2026-02-22T21:51:12.455 INFO:root:teuthology version: 1.2.4.dev9+gfa17720d0 2026-02-22T21:51:12.468 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com 2026-02-22T21:51:12.545 INFO:teuthology.run:Config: archive_path: /home/teuthworker/mnt/teuthology/yuriw-2026-02-22_19:35:38-rbd-wip-yuri6-testing-2026-02-22-1629-squid-distro-default-trial/64132 branch: wip-yuri6-testing-2026-02-22-1629-squid description: rbd/singleton/{all/admin_socket conf/{disable-pool-app} objectstore/bluestore-hybrid openstack supported-random-distro$/{ubuntu_latest}} email: yweinste@redhat.com first_in_suite: false flavor: default job_id: '64132' kernel: branch: distro kdb: true sha1: distro ktype: distro last_in_suite: false machine_type: trial name: yuriw-2026-02-22_19:35:38-rbd-wip-yuri6-testing-2026-02-22-1629-squid-distro-default-trial no_nested_subset: false openstack: - volumes: count: 2 size: 30 os_type: ubuntu os_version: '22.04' overrides: admin_socket: branch: wip-yuri6-testing-2026-02-22-1629-squid ceph: conf: global: mon warn on pool no app: false mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 osd: bluefs allocator: hybrid bluestore allocator: hybrid bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) sha1: f90f4eaebb28b619cf76e80cbbe997656da69579 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: f90f4eaebb28b619cf76e80cbbe997656da69579 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-yuri6-testing-2026-02-22-1629-squid sha1: f90f4eaebb28b619cf76e80cbbe997656da69579 owner: scheduled_yuriw@soko04.front.sepia.ceph.com priority: 69 repo: https://github.com/ceph/ceph-ci.git roles: - - mon.a - mgr.x - osd.0 - osd.1 - client.0 seed: 9196 sha1: f90f4eaebb28b619cf76e80cbbe997656da69579 sleep_before_teardown: 0 subset: 111/120000 suite: rbd suite_branch: wip-yuri6-testing-2026-02-22-1629-squid suite_path: /home/teuthworker/src/github.com_ceph_ceph-c_f90f4eaebb28b619cf76e80cbbe997656da69579/qa suite_relpath: qa suite_repo: https://github.com/ceph/ceph-ci.git suite_sha1: f90f4eaebb28b619cf76e80cbbe997656da69579 targets: trial093.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBB9rVJr06cLSWEqqEYI8bgLPlRtQV1IKTC6sJrOEtwq1qELr+VK5JNLrAFT0oSVDspQUdFV/QbfNmN1hG5eDx04= tasks: - install: null - ceph: fs: xfs - workunit: clients: all: - rbd/test_admin_socket.sh teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: fa17720d0088c3ac28e473468bfc79eeaff5cd38 timestamp: 2026-02-22_19:35:38 tube: trial user: yuriw verbose: true worker_log: /home/teuthworker/mnt/teuthology/worker_logs/dispatcher.trial.3318653 2026-02-22T21:51:12.545 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_ceph_ceph-c_f90f4eaebb28b619cf76e80cbbe997656da69579/qa; will attempt to use it 2026-02-22T21:51:12.545 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_ceph_ceph-c_f90f4eaebb28b619cf76e80cbbe997656da69579/qa/tasks 2026-02-22T21:51:12.545 INFO:teuthology.run_tasks:Running task internal.check_packages... 2026-02-22T21:51:12.547 INFO:teuthology.task.internal:Checking packages... 2026-02-22T21:51:12.547 INFO:teuthology.task.internal:Checking packages for os_type 'ubuntu', flavor 'default' and ceph hash 'f90f4eaebb28b619cf76e80cbbe997656da69579' 2026-02-22T21:51:12.547 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2026-02-22T21:51:12.547 INFO:teuthology.packaging:ref: None 2026-02-22T21:51:12.547 INFO:teuthology.packaging:tag: None 2026-02-22T21:51:12.547 INFO:teuthology.packaging:branch: wip-yuri6-testing-2026-02-22-1629-squid 2026-02-22T21:51:12.547 INFO:teuthology.packaging:sha1: f90f4eaebb28b619cf76e80cbbe997656da69579 2026-02-22T21:51:12.547 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&ref=wip-yuri6-testing-2026-02-22-1629-squid 2026-02-22T21:51:12.662 INFO:teuthology.task.internal:Found packages for ceph version 19.2.3-689-gf90f4eae-1jammy 2026-02-22T21:51:12.663 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2026-02-22T21:51:12.674 INFO:teuthology.task.internal:no buildpackages task found 2026-02-22T21:51:12.674 INFO:teuthology.run_tasks:Running task internal.save_config... 2026-02-22T21:51:12.688 INFO:teuthology.task.internal:Saving configuration 2026-02-22T21:51:12.697 INFO:teuthology.run_tasks:Running task internal.check_lock... 2026-02-22T21:51:12.708 INFO:teuthology.task.internal.check_lock:Checking locks... 2026-02-22T21:51:12.785 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'trial093.front.sepia.ceph.com', 'description': '/home/teuthworker/mnt/teuthology/yuriw-2026-02-22_19:35:38-rbd-wip-yuri6-testing-2026-02-22-1629-squid-distro-default-trial/64132', 'up': True, 'machine_type': 'trial', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2026-02-22 21:47:12.979639', 'locked_by': 'scheduled_yuriw@soko04.front.sepia.ceph.com', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIIK8zK5D2MsN+9I6s7dmW0SZBzc47d2zWA7bMCKqk3gp'} 2026-02-22T21:51:12.785 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2026-02-22T21:51:12.796 INFO:teuthology.task.internal:roles: ubuntu@trial093.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'osd.0', 'osd.1', 'client.0'] 2026-02-22T21:51:12.796 INFO:teuthology.run_tasks:Running task console_log... 2026-02-22T21:51:12.879 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x79a5783f7ec0>, signals=[15]) 2026-02-22T21:51:12.879 INFO:teuthology.run_tasks:Running task internal.connect... 2026-02-22T21:51:12.889 INFO:teuthology.task.internal:Opening connections... 2026-02-22T21:51:12.889 DEBUG:teuthology.task.internal:connecting to ubuntu@trial093.front.sepia.ceph.com 2026-02-22T21:51:12.890 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial093.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2026-02-22T21:51:12.954 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2026-02-22T21:51:12.966 DEBUG:teuthology.orchestra.run.trial093:> uname -m 2026-02-22T21:51:13.033 INFO:teuthology.orchestra.run.trial093.stdout:x86_64 2026-02-22T21:51:13.033 DEBUG:teuthology.orchestra.run.trial093:> cat /etc/os-release 2026-02-22T21:51:13.079 INFO:teuthology.orchestra.run.trial093.stdout:PRETTY_NAME="Ubuntu 22.04.5 LTS" 2026-02-22T21:51:13.079 INFO:teuthology.orchestra.run.trial093.stdout:NAME="Ubuntu" 2026-02-22T21:51:13.079 INFO:teuthology.orchestra.run.trial093.stdout:VERSION_ID="22.04" 2026-02-22T21:51:13.080 INFO:teuthology.orchestra.run.trial093.stdout:VERSION="22.04.5 LTS (Jammy Jellyfish)" 2026-02-22T21:51:13.080 INFO:teuthology.orchestra.run.trial093.stdout:VERSION_CODENAME=jammy 2026-02-22T21:51:13.080 INFO:teuthology.orchestra.run.trial093.stdout:ID=ubuntu 2026-02-22T21:51:13.080 INFO:teuthology.orchestra.run.trial093.stdout:ID_LIKE=debian 2026-02-22T21:51:13.080 INFO:teuthology.orchestra.run.trial093.stdout:HOME_URL="https://www.ubuntu.com/" 2026-02-22T21:51:13.080 INFO:teuthology.orchestra.run.trial093.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2026-02-22T21:51:13.080 INFO:teuthology.orchestra.run.trial093.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2026-02-22T21:51:13.080 INFO:teuthology.orchestra.run.trial093.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2026-02-22T21:51:13.080 INFO:teuthology.orchestra.run.trial093.stdout:UBUNTU_CODENAME=jammy 2026-02-22T21:51:13.081 INFO:teuthology.lock.ops:Updating trial093.front.sepia.ceph.com on lock server 2026-02-22T21:51:13.167 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2026-02-22T21:51:13.179 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2026-02-22T21:51:13.190 INFO:teuthology.task.internal:Checking for old test directory... 2026-02-22T21:51:13.190 DEBUG:teuthology.orchestra.run.trial093:> test '!' -e /home/ubuntu/cephtest 2026-02-22T21:51:13.194 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2026-02-22T21:51:13.204 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2026-02-22T21:51:13.204 DEBUG:teuthology.orchestra.run.trial093:> test -z $(ls -A /var/lib/ceph) 2026-02-22T21:51:13.239 INFO:teuthology.orchestra.run.trial093.stderr:ls: cannot access '/var/lib/ceph': No such file or directory 2026-02-22T21:51:13.239 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2026-02-22T21:51:13.311 INFO:teuthology.run_tasks:Running task kernel... 2026-02-22T21:51:13.330 INFO:teuthology.task.kernel:normalize config orig: {'branch': 'distro', 'kdb': True, 'sha1': 'distro'} 2026-02-22T21:51:13.330 INFO:teuthology.task.kernel:config {'mon.a': {'branch': 'distro', 'kdb': True, 'sha1': 'distro'}}, timeout 300 2026-02-22T21:51:13.330 DEBUG:teuthology.orchestra.run.trial093:> test -f /run/.containerenv -o -f /.dockerenv 2026-02-22T21:51:13.333 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-22T21:51:13.333 DEBUG:teuthology.orchestra.run.trial093:> uname -r 2026-02-22T21:51:13.383 INFO:teuthology.orchestra.run.trial093.stdout:5.15.0-170-generic 2026-02-22T21:51:13.383 INFO:teuthology.task.kernel:Running kernel on trial093: 5.15.0-170-generic 2026-02-22T21:51:13.383 DEBUG:teuthology.orchestra.run.trial093:> sudo apt-get clean 2026-02-22T21:51:13.442 DEBUG:teuthology.orchestra.run.trial093:> sudo apt-get update 2026-02-22T21:51:13.917 INFO:teuthology.orchestra.run.trial093.stdout:Hit:1 https://archive.ubuntu.com/ubuntu jammy InRelease 2026-02-22T21:51:14.002 INFO:teuthology.orchestra.run.trial093.stdout:Get:2 https://archive.ubuntu.com/ubuntu jammy-updates InRelease [128 kB] 2026-02-22T21:51:14.320 INFO:teuthology.orchestra.run.trial093.stdout:Get:3 https://archive.ubuntu.com/ubuntu jammy-security InRelease [129 kB] 2026-02-22T21:51:14.459 INFO:teuthology.orchestra.run.trial093.stdout:Get:4 https://archive.ubuntu.com/ubuntu jammy-backports InRelease [127 kB] 2026-02-22T21:51:14.590 INFO:teuthology.orchestra.run.trial093.stdout:Get:5 https://archive.ubuntu.com/ubuntu jammy-updates/main amd64 Packages [3245 kB] 2026-02-22T21:51:14.967 INFO:teuthology.orchestra.run.trial093.stdout:Get:6 https://archive.ubuntu.com/ubuntu jammy-updates/main Translation-en [495 kB] 2026-02-22T21:51:14.977 INFO:teuthology.orchestra.run.trial093.stdout:Get:7 https://archive.ubuntu.com/ubuntu jammy-updates/main amd64 c-n-f Metadata [19.1 kB] 2026-02-22T21:51:14.978 INFO:teuthology.orchestra.run.trial093.stdout:Get:8 https://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 Packages [1256 kB] 2026-02-22T21:51:15.011 INFO:teuthology.orchestra.run.trial093.stdout:Get:9 https://archive.ubuntu.com/ubuntu jammy-updates/universe Translation-en [314 kB] 2026-02-22T21:51:15.018 INFO:teuthology.orchestra.run.trial093.stdout:Get:10 https://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 c-n-f Metadata [30.3 kB] 2026-02-22T21:51:15.018 INFO:teuthology.orchestra.run.trial093.stdout:Get:11 https://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 Packages [5310 kB] 2026-02-22T21:51:15.166 INFO:teuthology.orchestra.run.trial093.stdout:Get:12 https://archive.ubuntu.com/ubuntu jammy-updates/restricted Translation-en [1005 kB] 2026-02-22T21:51:15.187 INFO:teuthology.orchestra.run.trial093.stdout:Get:13 https://archive.ubuntu.com/ubuntu jammy-security/main amd64 Packages [2983 kB] 2026-02-22T21:51:15.268 INFO:teuthology.orchestra.run.trial093.stdout:Get:14 https://archive.ubuntu.com/ubuntu jammy-security/main Translation-en [427 kB] 2026-02-22T21:51:15.277 INFO:teuthology.orchestra.run.trial093.stdout:Get:15 https://archive.ubuntu.com/ubuntu jammy-security/main amd64 c-n-f Metadata [14.1 kB] 2026-02-22T21:51:15.277 INFO:teuthology.orchestra.run.trial093.stdout:Get:16 https://archive.ubuntu.com/ubuntu jammy-security/universe amd64 Packages [1018 kB] 2026-02-22T21:51:15.309 INFO:teuthology.orchestra.run.trial093.stdout:Get:17 https://archive.ubuntu.com/ubuntu jammy-security/universe amd64 c-n-f Metadata [22.7 kB] 2026-02-22T21:51:15.309 INFO:teuthology.orchestra.run.trial093.stdout:Get:18 https://archive.ubuntu.com/ubuntu jammy-security/restricted amd64 Packages [5147 kB] 2026-02-22T21:51:15.450 INFO:teuthology.orchestra.run.trial093.stdout:Get:19 https://archive.ubuntu.com/ubuntu jammy-security/restricted Translation-en [979 kB] 2026-02-22T21:51:18.937 INFO:teuthology.orchestra.run.trial093.stdout:Fetched 22.6 MB in 3s (8096 kB/s) 2026-02-22T21:51:19.456 INFO:teuthology.orchestra.run.trial093.stdout:Reading package lists... 2026-02-22T21:51:19.470 DEBUG:teuthology.orchestra.run.trial093:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2026-02-22T21:51:19.503 INFO:teuthology.orchestra.run.trial093.stdout:Reading package lists... 2026-02-22T21:51:19.610 INFO:teuthology.orchestra.run.trial093.stdout:Building dependency tree... 2026-02-22T21:51:19.610 INFO:teuthology.orchestra.run.trial093.stdout:Reading state information... 2026-02-22T21:51:19.695 INFO:teuthology.orchestra.run.trial093.stdout:linux-image-generic is already the newest version (5.15.0.170.159). 2026-02-22T21:51:19.696 INFO:teuthology.orchestra.run.trial093.stdout:linux-image-generic set to manually installed. 2026-02-22T21:51:19.696 INFO:teuthology.orchestra.run.trial093.stdout:The following packages were automatically installed and are no longer required: 2026-02-22T21:51:19.696 INFO:teuthology.orchestra.run.trial093.stdout: kpartx libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 2026-02-22T21:51:19.696 INFO:teuthology.orchestra.run.trial093.stdout: libgfxdr0 libglusterfs0 libiscsi7 libpmemobj1 libsgutils2-2 sg3-utils 2026-02-22T21:51:19.696 INFO:teuthology.orchestra.run.trial093.stdout: sg3-utils-udev 2026-02-22T21:51:19.696 INFO:teuthology.orchestra.run.trial093.stdout:Use 'sudo apt autoremove' to remove them. 2026-02-22T21:51:19.715 INFO:teuthology.orchestra.run.trial093.stdout:0 upgraded, 0 newly installed, 0 to remove and 13 not upgraded. 2026-02-22T21:51:19.717 DEBUG:teuthology.orchestra.run.trial093:> dpkg -s linux-image-generic 2026-02-22T21:51:19.773 INFO:teuthology.orchestra.run.trial093.stdout:Package: linux-image-generic 2026-02-22T21:51:19.773 INFO:teuthology.orchestra.run.trial093.stdout:Status: install ok installed 2026-02-22T21:51:19.773 INFO:teuthology.orchestra.run.trial093.stdout:Priority: optional 2026-02-22T21:51:19.773 INFO:teuthology.orchestra.run.trial093.stdout:Section: kernel 2026-02-22T21:51:19.773 INFO:teuthology.orchestra.run.trial093.stdout:Installed-Size: 21 2026-02-22T21:51:19.773 INFO:teuthology.orchestra.run.trial093.stdout:Maintainer: Ubuntu Kernel Team 2026-02-22T21:51:19.773 INFO:teuthology.orchestra.run.trial093.stdout:Architecture: amd64 2026-02-22T21:51:19.773 INFO:teuthology.orchestra.run.trial093.stdout:Source: linux-meta 2026-02-22T21:51:19.773 INFO:teuthology.orchestra.run.trial093.stdout:Version: 5.15.0.170.159 2026-02-22T21:51:19.774 INFO:teuthology.orchestra.run.trial093.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.6), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-170), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.6) 2026-02-22T21:51:19.774 INFO:teuthology.orchestra.run.trial093.stdout:Depends: linux-image-5.15.0-170-generic, linux-modules-extra-5.15.0-170-generic, linux-firmware, intel-microcode, amd64-microcode 2026-02-22T21:51:19.774 INFO:teuthology.orchestra.run.trial093.stdout:Recommends: thermald 2026-02-22T21:51:19.774 INFO:teuthology.orchestra.run.trial093.stdout:Description: Generic Linux kernel image 2026-02-22T21:51:19.774 INFO:teuthology.orchestra.run.trial093.stdout: This package will always depend on the latest generic kernel image 2026-02-22T21:51:19.774 INFO:teuthology.orchestra.run.trial093.stdout: available. 2026-02-22T21:51:19.774 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2026-02-22T21:51:19.774 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2026-02-22T21:51:19.774 DEBUG:teuthology.orchestra.run.trial093:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2026-02-22T21:51:19.824 INFO:teuthology.orchestra.run.trial093.stdout:ttyS1 2026-02-22T21:51:19.858 DEBUG:teuthology.parallel:result is None 2026-02-22T21:51:19.858 INFO:teuthology.run_tasks:Running task internal.base... 2026-02-22T21:51:19.869 INFO:teuthology.task.internal:Creating test directory... 2026-02-22T21:51:19.870 DEBUG:teuthology.orchestra.run.trial093:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2026-02-22T21:51:19.904 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2026-02-22T21:51:19.916 INFO:teuthology.run_tasks:Running task internal.archive... 2026-02-22T21:51:19.928 INFO:teuthology.task.internal:Creating archive directory... 2026-02-22T21:51:19.928 DEBUG:teuthology.orchestra.run.trial093:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2026-02-22T21:51:19.953 INFO:teuthology.run_tasks:Running task internal.coredump... 2026-02-22T21:51:19.967 INFO:teuthology.task.internal:Enabling coredump saving... 2026-02-22T21:51:19.967 DEBUG:teuthology.orchestra.run.trial093:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2026-02-22T21:51:20.001 INFO:teuthology.orchestra.run.trial093.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-22T21:51:20.007 INFO:teuthology.orchestra.run.trial093.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-22T21:51:20.008 INFO:teuthology.run_tasks:Running task internal.sudo... 2026-02-22T21:51:20.020 INFO:teuthology.task.internal:Configuring sudo... 2026-02-22T21:51:20.020 DEBUG:teuthology.orchestra.run.trial093:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2026-02-22T21:51:20.064 INFO:teuthology.run_tasks:Running task internal.syslog... 2026-02-22T21:51:20.080 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2026-02-22T21:51:20.080 DEBUG:teuthology.orchestra.run.trial093:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2026-02-22T21:51:20.112 DEBUG:teuthology.orchestra.run.trial093:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2026-02-22T21:51:20.155 DEBUG:teuthology.orchestra.run.trial093:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2026-02-22T21:51:20.199 DEBUG:teuthology.orchestra.run.trial093:> set -ex 2026-02-22T21:51:20.199 DEBUG:teuthology.orchestra.run.trial093:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2026-02-22T21:51:20.247 DEBUG:teuthology.orchestra.run.trial093:> sudo service rsyslog restart 2026-02-22T21:51:20.304 INFO:teuthology.run_tasks:Running task internal.timer... 2026-02-22T21:51:20.317 INFO:teuthology.task.internal:Starting timer... 2026-02-22T21:51:20.317 INFO:teuthology.run_tasks:Running task pcp... 2026-02-22T21:51:20.337 INFO:teuthology.run_tasks:Running task selinux... 2026-02-22T21:51:20.354 DEBUG:teuthology.task.selinux:Excluding trial093: OS 'ubuntu' does not support SELinux 2026-02-22T21:51:20.355 DEBUG:teuthology.task.selinux:Getting current SELinux state 2026-02-22T21:51:20.355 DEBUG:teuthology.task.selinux:Existing SELinux modes: {} 2026-02-22T21:51:20.355 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2026-02-22T21:51:20.355 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2026-02-22T21:51:20.371 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2026-02-22T21:51:20.382 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2026-02-22T21:51:20.383 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit trial093.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2026-02-22T21:53:44.407 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@trial093.front.sepia.ceph.com')] 2026-02-22T21:53:44.410 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@trial093.front.sepia.ceph.com' 2026-02-22T21:53:44.413 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial093.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2026-02-22T21:53:44.522 DEBUG:teuthology.orchestra.run.trial093:> true 2026-02-22T21:53:44.773 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@trial093.front.sepia.ceph.com' 2026-02-22T21:53:44.774 INFO:teuthology.run_tasks:Running task clock... 2026-02-22T21:53:44.788 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2026-02-22T21:53:44.788 INFO:teuthology.orchestra.run:Running command with timeout 360 2026-02-22T21:53:44.789 DEBUG:teuthology.orchestra.run.trial093:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2026-02-22T21:53:44.837 INFO:teuthology.orchestra.run.trial093.stdout:22 Feb 21:53:45 ntpd[9429]: ntpd 4.2.8p15@1.3728-o Wed Feb 16 17:13:02 UTC 2022 (1): Starting 2026-02-22T21:53:44.838 INFO:teuthology.orchestra.run.trial093.stdout:22 Feb 21:53:45 ntpd[9429]: Command line: ntpd -gq 2026-02-22T21:53:44.838 INFO:teuthology.orchestra.run.trial093.stdout:22 Feb 21:53:45 ntpd[9429]: ---------------------------------------------------- 2026-02-22T21:53:44.838 INFO:teuthology.orchestra.run.trial093.stdout:22 Feb 21:53:45 ntpd[9429]: ntp-4 is maintained by Network Time Foundation, 2026-02-22T21:53:44.838 INFO:teuthology.orchestra.run.trial093.stdout:22 Feb 21:53:45 ntpd[9429]: Inc. (NTF), a non-profit 501(c)(3) public-benefit 2026-02-22T21:53:44.838 INFO:teuthology.orchestra.run.trial093.stdout:22 Feb 21:53:45 ntpd[9429]: corporation. Support and training for ntp-4 are 2026-02-22T21:53:44.838 INFO:teuthology.orchestra.run.trial093.stdout:22 Feb 21:53:45 ntpd[9429]: available at https://www.nwtime.org/support 2026-02-22T21:53:44.838 INFO:teuthology.orchestra.run.trial093.stdout:22 Feb 21:53:45 ntpd[9429]: ---------------------------------------------------- 2026-02-22T21:53:44.838 INFO:teuthology.orchestra.run.trial093.stdout:22 Feb 21:53:45 ntpd[9429]: proto: precision = 0.030 usec (-25) 2026-02-22T21:53:44.838 INFO:teuthology.orchestra.run.trial093.stdout:22 Feb 21:53:45 ntpd[9429]: basedate set to 2022-02-04 2026-02-22T21:53:44.838 INFO:teuthology.orchestra.run.trial093.stdout:22 Feb 21:53:45 ntpd[9429]: gps base set to 2022-02-06 (week 2196) 2026-02-22T21:53:44.839 INFO:teuthology.orchestra.run.trial093.stdout:22 Feb 21:53:45 ntpd[9429]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2026-02-22T21:53:44.839 INFO:teuthology.orchestra.run.trial093.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2026-02-22T21:53:44.839 INFO:teuthology.orchestra.run.trial093.stderr:restrict ::: KOD does nothing without LIMITED. 2026-02-22T21:53:44.840 INFO:teuthology.orchestra.run.trial093.stdout:22 Feb 21:53:45 ntpd[9429]: restrict ::: KOD does nothing without LIMITED. 2026-02-22T21:53:44.840 INFO:teuthology.orchestra.run.trial093.stdout:22 Feb 21:53:45 ntpd[9429]: Listen and drop on 0 v6wildcard [::]:123 2026-02-22T21:53:44.840 INFO:teuthology.orchestra.run.trial093.stdout:22 Feb 21:53:45 ntpd[9429]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2026-02-22T21:53:44.840 INFO:teuthology.orchestra.run.trial093.stdout:22 Feb 21:53:45 ntpd[9429]: Listen normally on 2 lo 127.0.0.1:123 2026-02-22T21:53:44.841 INFO:teuthology.orchestra.run.trial093.stdout:22 Feb 21:53:45 ntpd[9429]: Listen normally on 3 enp1s0f0 10.20.193.93:123 2026-02-22T21:53:44.841 INFO:teuthology.orchestra.run.trial093.stdout:22 Feb 21:53:45 ntpd[9429]: Listen normally on 4 lo [::1]:123 2026-02-22T21:53:44.841 INFO:teuthology.orchestra.run.trial093.stdout:22 Feb 21:53:45 ntpd[9429]: Listen normally on 5 enp1s0f0 [fe80::925a:8ff:fe77:62c6%2]:123 2026-02-22T21:53:44.841 INFO:teuthology.orchestra.run.trial093.stdout:22 Feb 21:53:45 ntpd[9429]: Listening on routing socket on fd #22 for interface updates 2026-02-22T21:53:47.837 INFO:teuthology.orchestra.run.trial093.stderr:22 Feb 21:53:48 ntpd[9429]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-22T21:53:47.838 INFO:teuthology.orchestra.run.trial093.stderr:22 Feb 21:53:48 ntpd[9429]: can't open /var/log/ntpstats/rawstats.20260222: Permission denied 2026-02-22T21:53:47.841 INFO:teuthology.orchestra.run.trial093.stderr:22 Feb 21:53:48 ntpd[9429]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-22T21:53:47.841 INFO:teuthology.orchestra.run.trial093.stderr:22 Feb 21:53:48 ntpd[9429]: can't open /var/log/ntpstats/rawstats.20260222: Permission denied 2026-02-22T21:53:47.841 INFO:teuthology.orchestra.run.trial093.stderr:22 Feb 21:53:48 ntpd[9429]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2026-02-22T21:53:47.841 INFO:teuthology.orchestra.run.trial093.stderr:22 Feb 21:53:48 ntpd[9429]: can't open /var/log/ntpstats/peerstats.20260222: Permission denied 2026-02-22T21:53:48.837 INFO:teuthology.orchestra.run.trial093.stderr:22 Feb 21:53:49 ntpd[9429]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-22T21:53:48.837 INFO:teuthology.orchestra.run.trial093.stderr:22 Feb 21:53:49 ntpd[9429]: can't open /var/log/ntpstats/rawstats.20260222: Permission denied 2026-02-22T21:53:48.841 INFO:teuthology.orchestra.run.trial093.stderr:22 Feb 21:53:49 ntpd[9429]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-22T21:53:48.841 INFO:teuthology.orchestra.run.trial093.stderr:22 Feb 21:53:49 ntpd[9429]: can't open /var/log/ntpstats/rawstats.20260222: Permission denied 2026-02-22T21:53:48.841 INFO:teuthology.orchestra.run.trial093.stderr:22 Feb 21:53:49 ntpd[9429]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2026-02-22T21:53:48.841 INFO:teuthology.orchestra.run.trial093.stderr:22 Feb 21:53:49 ntpd[9429]: can't open /var/log/ntpstats/peerstats.20260222: Permission denied 2026-02-22T21:53:49.837 INFO:teuthology.orchestra.run.trial093.stderr:22 Feb 21:53:50 ntpd[9429]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-22T21:53:49.837 INFO:teuthology.orchestra.run.trial093.stderr:22 Feb 21:53:50 ntpd[9429]: can't open /var/log/ntpstats/rawstats.20260222: Permission denied 2026-02-22T21:53:49.837 INFO:teuthology.orchestra.run.trial093.stderr:22 Feb 21:53:50 ntpd[9429]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-22T21:53:49.837 INFO:teuthology.orchestra.run.trial093.stderr:22 Feb 21:53:50 ntpd[9429]: can't open /var/log/ntpstats/rawstats.20260222: Permission denied 2026-02-22T21:53:49.840 INFO:teuthology.orchestra.run.trial093.stderr:22 Feb 21:53:50 ntpd[9429]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-22T21:53:49.840 INFO:teuthology.orchestra.run.trial093.stderr:22 Feb 21:53:50 ntpd[9429]: can't open /var/log/ntpstats/rawstats.20260222: Permission denied 2026-02-22T21:53:49.840 INFO:teuthology.orchestra.run.trial093.stderr:22 Feb 21:53:50 ntpd[9429]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2026-02-22T21:53:49.840 INFO:teuthology.orchestra.run.trial093.stderr:22 Feb 21:53:50 ntpd[9429]: can't open /var/log/ntpstats/peerstats.20260222: Permission denied 2026-02-22T21:53:49.912 INFO:teuthology.orchestra.run.trial093.stderr:22 Feb 21:53:50 ntpd[9429]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-22T21:53:49.912 INFO:teuthology.orchestra.run.trial093.stderr:22 Feb 21:53:50 ntpd[9429]: can't open /var/log/ntpstats/rawstats.20260222: Permission denied 2026-02-22T21:53:49.912 INFO:teuthology.orchestra.run.trial093.stderr:22 Feb 21:53:50 ntpd[9429]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2026-02-22T21:53:49.912 INFO:teuthology.orchestra.run.trial093.stderr:22 Feb 21:53:50 ntpd[9429]: can't open /var/log/ntpstats/peerstats.20260222: Permission denied 2026-02-22T21:53:50.838 INFO:teuthology.orchestra.run.trial093.stderr:22 Feb 21:53:51 ntpd[9429]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-22T21:53:50.838 INFO:teuthology.orchestra.run.trial093.stderr:22 Feb 21:53:51 ntpd[9429]: can't open /var/log/ntpstats/rawstats.20260222: Permission denied 2026-02-22T21:53:50.841 INFO:teuthology.orchestra.run.trial093.stderr:22 Feb 21:53:51 ntpd[9429]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-22T21:53:50.842 INFO:teuthology.orchestra.run.trial093.stderr:22 Feb 21:53:51 ntpd[9429]: can't open /var/log/ntpstats/rawstats.20260222: Permission denied 2026-02-22T21:53:50.842 INFO:teuthology.orchestra.run.trial093.stderr:22 Feb 21:53:51 ntpd[9429]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2026-02-22T21:53:50.842 INFO:teuthology.orchestra.run.trial093.stderr:22 Feb 21:53:51 ntpd[9429]: can't open /var/log/ntpstats/peerstats.20260222: Permission denied 2026-02-22T21:53:51.837 INFO:teuthology.orchestra.run.trial093.stderr:22 Feb 21:53:52 ntpd[9429]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-22T21:53:51.838 INFO:teuthology.orchestra.run.trial093.stderr:22 Feb 21:53:52 ntpd[9429]: can't open /var/log/ntpstats/rawstats.20260222: Permission denied 2026-02-22T21:53:51.838 INFO:teuthology.orchestra.run.trial093.stderr:22 Feb 21:53:52 ntpd[9429]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-22T21:53:51.838 INFO:teuthology.orchestra.run.trial093.stderr:22 Feb 21:53:52 ntpd[9429]: can't open /var/log/ntpstats/rawstats.20260222: Permission denied 2026-02-22T21:53:51.840 INFO:teuthology.orchestra.run.trial093.stderr:22 Feb 21:53:52 ntpd[9429]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-22T21:53:51.840 INFO:teuthology.orchestra.run.trial093.stderr:22 Feb 21:53:52 ntpd[9429]: can't open /var/log/ntpstats/rawstats.20260222: Permission denied 2026-02-22T21:53:51.841 INFO:teuthology.orchestra.run.trial093.stderr:22 Feb 21:53:52 ntpd[9429]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2026-02-22T21:53:51.841 INFO:teuthology.orchestra.run.trial093.stderr:22 Feb 21:53:52 ntpd[9429]: can't open /var/log/ntpstats/peerstats.20260222: Permission denied 2026-02-22T21:53:51.912 INFO:teuthology.orchestra.run.trial093.stderr:22 Feb 21:53:52 ntpd[9429]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-22T21:53:51.912 INFO:teuthology.orchestra.run.trial093.stderr:22 Feb 21:53:52 ntpd[9429]: can't open /var/log/ntpstats/rawstats.20260222: Permission denied 2026-02-22T21:53:51.912 INFO:teuthology.orchestra.run.trial093.stderr:22 Feb 21:53:52 ntpd[9429]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2026-02-22T21:53:51.912 INFO:teuthology.orchestra.run.trial093.stderr:22 Feb 21:53:52 ntpd[9429]: can't open /var/log/ntpstats/peerstats.20260222: Permission denied 2026-02-22T21:53:52.837 INFO:teuthology.orchestra.run.trial093.stderr:22 Feb 21:53:53 ntpd[9429]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-22T21:53:52.837 INFO:teuthology.orchestra.run.trial093.stderr:22 Feb 21:53:53 ntpd[9429]: can't open /var/log/ntpstats/rawstats.20260222: Permission denied 2026-02-22T21:53:52.841 INFO:teuthology.orchestra.run.trial093.stderr:22 Feb 21:53:53 ntpd[9429]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-22T21:53:52.841 INFO:teuthology.orchestra.run.trial093.stderr:22 Feb 21:53:53 ntpd[9429]: can't open /var/log/ntpstats/rawstats.20260222: Permission denied 2026-02-22T21:53:52.841 INFO:teuthology.orchestra.run.trial093.stderr:22 Feb 21:53:53 ntpd[9429]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2026-02-22T21:53:52.841 INFO:teuthology.orchestra.run.trial093.stderr:22 Feb 21:53:53 ntpd[9429]: can't open /var/log/ntpstats/peerstats.20260222: Permission denied 2026-02-22T21:53:53.837 INFO:teuthology.orchestra.run.trial093.stderr:22 Feb 21:53:54 ntpd[9429]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-22T21:53:53.838 INFO:teuthology.orchestra.run.trial093.stderr:22 Feb 21:53:54 ntpd[9429]: can't open /var/log/ntpstats/rawstats.20260222: Permission denied 2026-02-22T21:53:53.838 INFO:teuthology.orchestra.run.trial093.stderr:22 Feb 21:53:54 ntpd[9429]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-22T21:53:53.838 INFO:teuthology.orchestra.run.trial093.stderr:22 Feb 21:53:54 ntpd[9429]: can't open /var/log/ntpstats/rawstats.20260222: Permission denied 2026-02-22T21:53:53.840 INFO:teuthology.orchestra.run.trial093.stderr:22 Feb 21:53:54 ntpd[9429]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2026-02-22T21:53:53.840 INFO:teuthology.orchestra.run.trial093.stderr:22 Feb 21:53:54 ntpd[9429]: can't open /var/log/ntpstats/rawstats.20260222: Permission denied 2026-02-22T21:53:53.841 INFO:teuthology.orchestra.run.trial093.stderr:22 Feb 21:53:54 ntpd[9429]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2026-02-22T21:53:53.841 INFO:teuthology.orchestra.run.trial093.stderr:22 Feb 21:53:54 ntpd[9429]: can't open /var/log/ntpstats/peerstats.20260222: Permission denied 2026-02-22T21:53:53.841 INFO:teuthology.orchestra.run.trial093.stdout:22 Feb 21:53:53 ntpd[9429]: ntpd: time set -0.216501 s 2026-02-22T21:53:53.841 INFO:teuthology.orchestra.run.trial093.stdout:ntpd: time set -0.216501s 2026-02-22T21:53:53.842 INFO:teuthology.orchestra.run.trial093.stderr:22 Feb 21:53:53 ntpd[9429]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2026-02-22T21:53:53.842 INFO:teuthology.orchestra.run.trial093.stderr:22 Feb 21:53:53 ntpd[9429]: can't open /var/log/ntpstats/loopstats.20260222: Permission denied 2026-02-22T21:53:53.866 INFO:teuthology.orchestra.run.trial093.stdout:No association ID's returned 2026-02-22T21:53:53.866 INFO:teuthology.run_tasks:Running task install... 2026-02-22T21:53:53.881 DEBUG:teuthology.task.install:project ceph 2026-02-22T21:53:53.881 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'f90f4eaebb28b619cf76e80cbbe997656da69579'}} 2026-02-22T21:53:53.882 DEBUG:teuthology.task.install:config {'flavor': 'default', 'sha1': 'f90f4eaebb28b619cf76e80cbbe997656da69579'} 2026-02-22T21:53:53.882 INFO:teuthology.task.install:Using flavor: default 2026-02-22T21:53:53.890 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2026-02-22T21:53:53.890 INFO:teuthology.task.install:extra packages: [] 2026-02-22T21:53:53.891 DEBUG:teuthology.orchestra.run.trial093:> sudo apt-key list | grep Ceph 2026-02-22T21:53:53.966 INFO:teuthology.orchestra.run.trial093.stderr:Warning: apt-key is deprecated. Manage keyring files in trusted.gpg.d instead (see apt-key(8)). 2026-02-22T21:53:53.995 INFO:teuthology.orchestra.run.trial093.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2026-02-22T21:53:53.995 INFO:teuthology.orchestra.run.trial093.stdout:uid [ unknown] Ceph.com (release key) 2026-02-22T21:53:53.996 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse on remote deb x86_64 2026-02-22T21:53:53.997 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=f90f4eaebb28b619cf76e80cbbe997656da69579 2026-02-22T21:53:54.168 INFO:teuthology.task.install.deb:Pulling from https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2026-02-22-1629-squid/f90f4eaebb28b619cf76e80cbbe997656da69579/ubuntu/jammy/flavors/default/ 2026-02-22T21:53:54.168 INFO:teuthology.task.install.deb:Package version is 19.2.3-689-gf90f4eae-1jammy 2026-02-22T21:53:54.299 DEBUG:teuthology.orchestra.run.trial093:> set -ex 2026-02-22T21:53:54.299 DEBUG:teuthology.orchestra.run.trial093:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2026-02-22T21:53:54.309 DEBUG:teuthology.orchestra.run.trial093:> sudo apt-get update 2026-02-22T21:53:54.460 INFO:teuthology.orchestra.run.trial093.stdout:Ign:1 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2026-02-22-1629-squid/f90f4eaebb28b619cf76e80cbbe997656da69579/ubuntu/jammy/flavors/default jammy InRelease 2026-02-22T21:53:54.471 INFO:teuthology.orchestra.run.trial093.stdout:Get:2 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2026-02-22-1629-squid/f90f4eaebb28b619cf76e80cbbe997656da69579/ubuntu/jammy/flavors/default jammy Release [7662 B] 2026-02-22T21:53:54.483 INFO:teuthology.orchestra.run.trial093.stdout:Ign:3 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2026-02-22-1629-squid/f90f4eaebb28b619cf76e80cbbe997656da69579/ubuntu/jammy/flavors/default jammy Release.gpg 2026-02-22T21:53:54.495 INFO:teuthology.orchestra.run.trial093.stdout:Get:4 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2026-02-22-1629-squid/f90f4eaebb28b619cf76e80cbbe997656da69579/ubuntu/jammy/flavors/default jammy/main amd64 Packages [18.1 kB] 2026-02-22T21:53:54.808 INFO:teuthology.orchestra.run.trial093.stdout:Hit:5 https://archive.ubuntu.com/ubuntu jammy InRelease 2026-02-22T21:53:54.889 INFO:teuthology.orchestra.run.trial093.stdout:Hit:6 https://archive.ubuntu.com/ubuntu jammy-updates InRelease 2026-02-22T21:53:54.977 INFO:teuthology.orchestra.run.trial093.stdout:Hit:7 https://archive.ubuntu.com/ubuntu jammy-security InRelease 2026-02-22T21:53:55.065 INFO:teuthology.orchestra.run.trial093.stdout:Hit:8 https://archive.ubuntu.com/ubuntu jammy-backports InRelease 2026-02-22T21:53:55.207 INFO:teuthology.orchestra.run.trial093.stdout:Fetched 25.8 kB in 1s (34.9 kB/s) 2026-02-22T21:53:55.732 INFO:teuthology.orchestra.run.trial093.stdout:Reading package lists... 2026-02-22T21:53:55.746 DEBUG:teuthology.orchestra.run.trial093:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=19.2.3-689-gf90f4eae-1jammy cephadm=19.2.3-689-gf90f4eae-1jammy ceph-mds=19.2.3-689-gf90f4eae-1jammy ceph-mgr=19.2.3-689-gf90f4eae-1jammy ceph-common=19.2.3-689-gf90f4eae-1jammy ceph-fuse=19.2.3-689-gf90f4eae-1jammy ceph-test=19.2.3-689-gf90f4eae-1jammy ceph-volume=19.2.3-689-gf90f4eae-1jammy radosgw=19.2.3-689-gf90f4eae-1jammy python3-rados=19.2.3-689-gf90f4eae-1jammy python3-rgw=19.2.3-689-gf90f4eae-1jammy python3-cephfs=19.2.3-689-gf90f4eae-1jammy python3-rbd=19.2.3-689-gf90f4eae-1jammy libcephfs2=19.2.3-689-gf90f4eae-1jammy libcephfs-dev=19.2.3-689-gf90f4eae-1jammy librados2=19.2.3-689-gf90f4eae-1jammy librbd1=19.2.3-689-gf90f4eae-1jammy rbd-fuse=19.2.3-689-gf90f4eae-1jammy 2026-02-22T21:53:55.780 INFO:teuthology.orchestra.run.trial093.stdout:Reading package lists... 2026-02-22T21:53:55.891 INFO:teuthology.orchestra.run.trial093.stdout:Building dependency tree... 2026-02-22T21:53:55.892 INFO:teuthology.orchestra.run.trial093.stdout:Reading state information... 2026-02-22T21:53:55.985 INFO:teuthology.orchestra.run.trial093.stdout:The following packages were automatically installed and are no longer required: 2026-02-22T21:53:55.986 INFO:teuthology.orchestra.run.trial093.stdout: kpartx libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 2026-02-22T21:53:55.986 INFO:teuthology.orchestra.run.trial093.stdout: libgfxdr0 libglusterfs0 libiscsi7 libpmemobj1 libsgutils2-2 sg3-utils 2026-02-22T21:53:55.986 INFO:teuthology.orchestra.run.trial093.stdout: sg3-utils-udev 2026-02-22T21:53:55.986 INFO:teuthology.orchestra.run.trial093.stdout:Use 'sudo apt autoremove' to remove them. 2026-02-22T21:53:55.987 INFO:teuthology.orchestra.run.trial093.stdout:The following additional packages will be installed: 2026-02-22T21:53:55.987 INFO:teuthology.orchestra.run.trial093.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2026-02-22T21:53:55.987 INFO:teuthology.orchestra.run.trial093.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd jq 2026-02-22T21:53:55.987 INFO:teuthology.orchestra.run.trial093.stdout: libdouble-conversion3 libfuse2 libjq1 liblttng-ust1 liblua5.3-dev libnbd0 2026-02-22T21:53:55.987 INFO:teuthology.orchestra.run.trial093.stdout: liboath0 libonig5 libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 2026-02-22T21:53:55.987 INFO:teuthology.orchestra.run.trial093.stdout: libradosstriper1 librdkafka1 libreadline-dev librgw2 libsqlite3-mod-ceph 2026-02-22T21:53:55.988 INFO:teuthology.orchestra.run.trial093.stdout: libthrift-0.16.0 lua-any lua-sec lua-socket lua5.1 luarocks nvme-cli 2026-02-22T21:53:55.988 INFO:teuthology.orchestra.run.trial093.stdout: pkg-config python-asyncssh-doc python-pastedeploy-tpl python3-asyncssh 2026-02-22T21:53:55.988 INFO:teuthology.orchestra.run.trial093.stdout: python3-cachetools python3-ceph-argparse python3-ceph-common python3-cheroot 2026-02-22T21:53:55.988 INFO:teuthology.orchestra.run.trial093.stdout: python3-cherrypy3 python3-google-auth python3-iniconfig 2026-02-22T21:53:55.988 INFO:teuthology.orchestra.run.trial093.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2026-02-22T21:53:55.989 INFO:teuthology.orchestra.run.trial093.stdout: python3-jaraco.text python3-joblib python3-kubernetes python3-logutils 2026-02-22T21:53:55.989 INFO:teuthology.orchestra.run.trial093.stdout: python3-mako python3-natsort python3-paste python3-pastedeploy 2026-02-22T21:53:55.989 INFO:teuthology.orchestra.run.trial093.stdout: python3-pastescript python3-pecan python3-pluggy python3-portend 2026-02-22T21:53:55.989 INFO:teuthology.orchestra.run.trial093.stdout: python3-prettytable python3-psutil python3-py python3-pygments 2026-02-22T21:53:55.989 INFO:teuthology.orchestra.run.trial093.stdout: python3-pyinotify python3-pytest python3-repoze.lru 2026-02-22T21:53:55.989 INFO:teuthology.orchestra.run.trial093.stdout: python3-requests-oauthlib python3-routes python3-rsa python3-simplegeneric 2026-02-22T21:53:55.990 INFO:teuthology.orchestra.run.trial093.stdout: python3-simplejson python3-singledispatch python3-sklearn 2026-02-22T21:53:55.990 INFO:teuthology.orchestra.run.trial093.stdout: python3-sklearn-lib python3-tempita python3-tempora python3-threadpoolctl 2026-02-22T21:53:55.990 INFO:teuthology.orchestra.run.trial093.stdout: python3-toml python3-waitress python3-wcwidth python3-webob 2026-02-22T21:53:55.990 INFO:teuthology.orchestra.run.trial093.stdout: python3-websocket python3-webtest python3-werkzeug python3-zc.lockfile 2026-02-22T21:53:55.990 INFO:teuthology.orchestra.run.trial093.stdout: qttranslations5-l10n socat unzip xmlstarlet zip 2026-02-22T21:53:55.990 INFO:teuthology.orchestra.run.trial093.stdout:Suggested packages: 2026-02-22T21:53:55.991 INFO:teuthology.orchestra.run.trial093.stdout: python3-influxdb readline-doc python3-beaker python-mako-doc 2026-02-22T21:53:55.991 INFO:teuthology.orchestra.run.trial093.stdout: python-natsort-doc httpd-wsgi libapache2-mod-python libapache2-mod-scgi 2026-02-22T21:53:55.991 INFO:teuthology.orchestra.run.trial093.stdout: libjs-mochikit python-pecan-doc python-psutil-doc subversion 2026-02-22T21:53:55.991 INFO:teuthology.orchestra.run.trial093.stdout: python-pygments-doc ttf-bitstream-vera python-pyinotify-doc python3-dap 2026-02-22T21:53:55.991 INFO:teuthology.orchestra.run.trial093.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2026-02-22T21:53:55.991 INFO:teuthology.orchestra.run.trial093.stdout: python-webtest-doc python-werkzeug-doc python3-watchdog 2026-02-22T21:53:55.992 INFO:teuthology.orchestra.run.trial093.stdout:Recommended packages: 2026-02-22T21:53:55.992 INFO:teuthology.orchestra.run.trial093.stdout: btrfs-tools 2026-02-22T21:53:56.019 INFO:teuthology.orchestra.run.trial093.stdout:The following NEW packages will be installed: 2026-02-22T21:53:56.019 INFO:teuthology.orchestra.run.trial093.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2026-02-22T21:53:56.020 INFO:teuthology.orchestra.run.trial093.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2026-02-22T21:53:56.020 INFO:teuthology.orchestra.run.trial093.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm jq 2026-02-22T21:53:56.020 INFO:teuthology.orchestra.run.trial093.stdout: libcephfs-dev libcephfs2 libdouble-conversion3 libfuse2 libjq1 liblttng-ust1 2026-02-22T21:53:56.020 INFO:teuthology.orchestra.run.trial093.stdout: liblua5.3-dev libnbd0 liboath0 libonig5 libpcre2-16-0 libqt5core5a 2026-02-22T21:53:56.020 INFO:teuthology.orchestra.run.trial093.stdout: libqt5dbus5 libqt5network5 librados2 libradosstriper1 librbd1 librdkafka1 2026-02-22T21:53:56.020 INFO:teuthology.orchestra.run.trial093.stdout: libreadline-dev librgw2 libsqlite3-mod-ceph libthrift-0.16.0 lua-any lua-sec 2026-02-22T21:53:56.020 INFO:teuthology.orchestra.run.trial093.stdout: lua-socket lua5.1 luarocks nvme-cli pkg-config python-asyncssh-doc 2026-02-22T21:53:56.020 INFO:teuthology.orchestra.run.trial093.stdout: python-pastedeploy-tpl python3-asyncssh python3-cachetools 2026-02-22T21:53:56.020 INFO:teuthology.orchestra.run.trial093.stdout: python3-ceph-argparse python3-ceph-common python3-cephfs python3-cheroot 2026-02-22T21:53:56.020 INFO:teuthology.orchestra.run.trial093.stdout: python3-cherrypy3 python3-google-auth python3-iniconfig 2026-02-22T21:53:56.020 INFO:teuthology.orchestra.run.trial093.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2026-02-22T21:53:56.021 INFO:teuthology.orchestra.run.trial093.stdout: python3-jaraco.text python3-joblib python3-kubernetes python3-logutils 2026-02-22T21:53:56.021 INFO:teuthology.orchestra.run.trial093.stdout: python3-mako python3-natsort python3-paste python3-pastedeploy 2026-02-22T21:53:56.021 INFO:teuthology.orchestra.run.trial093.stdout: python3-pastescript python3-pecan python3-pluggy python3-portend 2026-02-22T21:53:56.021 INFO:teuthology.orchestra.run.trial093.stdout: python3-prettytable python3-psutil python3-py python3-pygments 2026-02-22T21:53:56.021 INFO:teuthology.orchestra.run.trial093.stdout: python3-pyinotify python3-pytest python3-rados python3-rbd 2026-02-22T21:53:56.021 INFO:teuthology.orchestra.run.trial093.stdout: python3-repoze.lru python3-requests-oauthlib python3-rgw python3-routes 2026-02-22T21:53:56.021 INFO:teuthology.orchestra.run.trial093.stdout: python3-rsa python3-simplegeneric python3-simplejson python3-singledispatch 2026-02-22T21:53:56.021 INFO:teuthology.orchestra.run.trial093.stdout: python3-sklearn python3-sklearn-lib python3-tempita python3-tempora 2026-02-22T21:53:56.021 INFO:teuthology.orchestra.run.trial093.stdout: python3-threadpoolctl python3-toml python3-waitress python3-wcwidth 2026-02-22T21:53:56.021 INFO:teuthology.orchestra.run.trial093.stdout: python3-webob python3-websocket python3-webtest python3-werkzeug 2026-02-22T21:53:56.021 INFO:teuthology.orchestra.run.trial093.stdout: python3-zc.lockfile qttranslations5-l10n radosgw rbd-fuse socat unzip 2026-02-22T21:53:56.021 INFO:teuthology.orchestra.run.trial093.stdout: xmlstarlet zip 2026-02-22T21:53:56.098 INFO:teuthology.orchestra.run.trial093.stdout:0 upgraded, 108 newly installed, 0 to remove and 13 not upgraded. 2026-02-22T21:53:56.099 INFO:teuthology.orchestra.run.trial093.stdout:Need to get 178 MB of archives. 2026-02-22T21:53:56.099 INFO:teuthology.orchestra.run.trial093.stdout:After this operation, 809 MB of additional disk space will be used. 2026-02-22T21:53:56.099 INFO:teuthology.orchestra.run.trial093.stdout:Get:1 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2026-02-22-1629-squid/f90f4eaebb28b619cf76e80cbbe997656da69579/ubuntu/jammy/flavors/default jammy/main amd64 librados2 amd64 19.2.3-689-gf90f4eae-1jammy [3597 kB] 2026-02-22T21:53:56.201 INFO:teuthology.orchestra.run.trial093.stdout:Get:2 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2026-02-22-1629-squid/f90f4eaebb28b619cf76e80cbbe997656da69579/ubuntu/jammy/flavors/default jammy/main amd64 librbd1 amd64 19.2.3-689-gf90f4eae-1jammy [3256 kB] 2026-02-22T21:53:56.246 INFO:teuthology.orchestra.run.trial093.stdout:Get:3 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2026-02-22-1629-squid/f90f4eaebb28b619cf76e80cbbe997656da69579/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs2 amd64 19.2.3-689-gf90f4eae-1jammy [979 kB] 2026-02-22T21:53:56.261 INFO:teuthology.orchestra.run.trial093.stdout:Get:4 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2026-02-22-1629-squid/f90f4eaebb28b619cf76e80cbbe997656da69579/ubuntu/jammy/flavors/default jammy/main amd64 python3-rados amd64 19.2.3-689-gf90f4eae-1jammy [357 kB] 2026-02-22T21:53:56.268 INFO:teuthology.orchestra.run.trial093.stdout:Get:5 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2026-02-22-1629-squid/f90f4eaebb28b619cf76e80cbbe997656da69579/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-argparse all 19.2.3-689-gf90f4eae-1jammy [32.9 kB] 2026-02-22T21:53:56.268 INFO:teuthology.orchestra.run.trial093.stdout:Get:6 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2026-02-22-1629-squid/f90f4eaebb28b619cf76e80cbbe997656da69579/ubuntu/jammy/flavors/default jammy/main amd64 python3-cephfs amd64 19.2.3-689-gf90f4eae-1jammy [184 kB] 2026-02-22T21:53:56.271 INFO:teuthology.orchestra.run.trial093.stdout:Get:7 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2026-02-22-1629-squid/f90f4eaebb28b619cf76e80cbbe997656da69579/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-common all 19.2.3-689-gf90f4eae-1jammy [70.0 kB] 2026-02-22T21:53:56.272 INFO:teuthology.orchestra.run.trial093.stdout:Get:8 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2026-02-22-1629-squid/f90f4eaebb28b619cf76e80cbbe997656da69579/ubuntu/jammy/flavors/default jammy/main amd64 python3-rbd amd64 19.2.3-689-gf90f4eae-1jammy [335 kB] 2026-02-22T21:53:56.277 INFO:teuthology.orchestra.run.trial093.stdout:Get:9 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2026-02-22-1629-squid/f90f4eaebb28b619cf76e80cbbe997656da69579/ubuntu/jammy/flavors/default jammy/main amd64 librgw2 amd64 19.2.3-689-gf90f4eae-1jammy [6935 kB] 2026-02-22T21:53:56.371 INFO:teuthology.orchestra.run.trial093.stdout:Get:10 https://archive.ubuntu.com/ubuntu jammy/main amd64 liblttng-ust1 amd64 2.13.1-1ubuntu1 [190 kB] 2026-02-22T21:53:56.392 INFO:teuthology.orchestra.run.trial093.stdout:Get:11 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2026-02-22-1629-squid/f90f4eaebb28b619cf76e80cbbe997656da69579/ubuntu/jammy/flavors/default jammy/main amd64 python3-rgw amd64 19.2.3-689-gf90f4eae-1jammy [112 kB] 2026-02-22T21:53:56.393 INFO:teuthology.orchestra.run.trial093.stdout:Get:12 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2026-02-22-1629-squid/f90f4eaebb28b619cf76e80cbbe997656da69579/ubuntu/jammy/flavors/default jammy/main amd64 libradosstriper1 amd64 19.2.3-689-gf90f4eae-1jammy [470 kB] 2026-02-22T21:53:56.400 INFO:teuthology.orchestra.run.trial093.stdout:Get:13 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2026-02-22-1629-squid/f90f4eaebb28b619cf76e80cbbe997656da69579/ubuntu/jammy/flavors/default jammy/main amd64 ceph-common amd64 19.2.3-689-gf90f4eae-1jammy [26.5 MB] 2026-02-22T21:53:56.709 INFO:teuthology.orchestra.run.trial093.stdout:Get:14 https://archive.ubuntu.com/ubuntu jammy/universe amd64 libdouble-conversion3 amd64 3.1.7-4 [39.0 kB] 2026-02-22T21:53:56.720 INFO:teuthology.orchestra.run.trial093.stdout:Get:15 https://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpcre2-16-0 amd64 10.39-3ubuntu0.1 [203 kB] 2026-02-22T21:53:56.790 INFO:teuthology.orchestra.run.trial093.stdout:Get:16 https://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5core5a amd64 5.15.3+dfsg-2ubuntu0.2 [2006 kB] 2026-02-22T21:53:56.838 INFO:teuthology.orchestra.run.trial093.stdout:Get:17 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2026-02-22-1629-squid/f90f4eaebb28b619cf76e80cbbe997656da69579/ubuntu/jammy/flavors/default jammy/main amd64 ceph-base amd64 19.2.3-689-gf90f4eae-1jammy [5178 kB] 2026-02-22T21:53:56.920 INFO:teuthology.orchestra.run.trial093.stdout:Get:18 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2026-02-22-1629-squid/f90f4eaebb28b619cf76e80cbbe997656da69579/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-modules-core all 19.2.3-689-gf90f4eae-1jammy [248 kB] 2026-02-22T21:53:56.922 INFO:teuthology.orchestra.run.trial093.stdout:Get:19 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2026-02-22-1629-squid/f90f4eaebb28b619cf76e80cbbe997656da69579/ubuntu/jammy/flavors/default jammy/main amd64 libsqlite3-mod-ceph amd64 19.2.3-689-gf90f4eae-1jammy [125 kB] 2026-02-22T21:53:56.925 INFO:teuthology.orchestra.run.trial093.stdout:Get:20 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2026-02-22-1629-squid/f90f4eaebb28b619cf76e80cbbe997656da69579/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr amd64 19.2.3-689-gf90f4eae-1jammy [1081 kB] 2026-02-22T21:53:56.943 INFO:teuthology.orchestra.run.trial093.stdout:Get:21 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2026-02-22-1629-squid/f90f4eaebb28b619cf76e80cbbe997656da69579/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mon amd64 19.2.3-689-gf90f4eae-1jammy [6239 kB] 2026-02-22T21:53:56.992 INFO:teuthology.orchestra.run.trial093.stdout:Get:22 https://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5dbus5 amd64 5.15.3+dfsg-2ubuntu0.2 [222 kB] 2026-02-22T21:53:57.003 INFO:teuthology.orchestra.run.trial093.stdout:Get:23 https://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5network5 amd64 5.15.3+dfsg-2ubuntu0.2 [731 kB] 2026-02-22T21:53:57.029 INFO:teuthology.orchestra.run.trial093.stdout:Get:24 https://archive.ubuntu.com/ubuntu jammy/universe amd64 libthrift-0.16.0 amd64 0.16.0-2 [267 kB] 2026-02-22T21:53:57.038 INFO:teuthology.orchestra.run.trial093.stdout:Get:25 https://archive.ubuntu.com/ubuntu jammy/universe amd64 libnbd0 amd64 1.10.5-1 [71.3 kB] 2026-02-22T21:53:57.040 INFO:teuthology.orchestra.run.trial093.stdout:Get:26 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-wcwidth all 0.2.5+dfsg1-1 [21.9 kB] 2026-02-22T21:53:57.040 INFO:teuthology.orchestra.run.trial093.stdout:Get:27 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-prettytable all 2.5.0-2 [31.3 kB] 2026-02-22T21:53:57.041 INFO:teuthology.orchestra.run.trial093.stdout:Get:28 https://archive.ubuntu.com/ubuntu jammy/universe amd64 librdkafka1 amd64 1.8.0-1build1 [633 kB] 2026-02-22T21:53:57.045 INFO:teuthology.orchestra.run.trial093.stdout:Get:29 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2026-02-22-1629-squid/f90f4eaebb28b619cf76e80cbbe997656da69579/ubuntu/jammy/flavors/default jammy/main amd64 ceph-osd amd64 19.2.3-689-gf90f4eae-1jammy [23.0 MB] 2026-02-22T21:53:57.058 INFO:teuthology.orchestra.run.trial093.stdout:Get:30 https://archive.ubuntu.com/ubuntu jammy/main amd64 libreadline-dev amd64 8.1.2-1 [166 kB] 2026-02-22T21:53:57.062 INFO:teuthology.orchestra.run.trial093.stdout:Get:31 https://archive.ubuntu.com/ubuntu jammy/main amd64 liblua5.3-dev amd64 5.3.6-1build1 [167 kB] 2026-02-22T21:53:57.065 INFO:teuthology.orchestra.run.trial093.stdout:Get:32 https://archive.ubuntu.com/ubuntu jammy/universe amd64 lua5.1 amd64 5.1.5-8.1build4 [94.6 kB] 2026-02-22T21:53:57.134 INFO:teuthology.orchestra.run.trial093.stdout:Get:33 https://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-any all 27ubuntu1 [5034 B] 2026-02-22T21:53:57.135 INFO:teuthology.orchestra.run.trial093.stdout:Get:34 https://archive.ubuntu.com/ubuntu jammy/main amd64 zip amd64 3.0-12build2 [176 kB] 2026-02-22T21:53:57.136 INFO:teuthology.orchestra.run.trial093.stdout:Get:35 https://archive.ubuntu.com/ubuntu jammy-updates/main amd64 unzip amd64 6.0-26ubuntu3.2 [175 kB] 2026-02-22T21:53:57.138 INFO:teuthology.orchestra.run.trial093.stdout:Get:36 https://archive.ubuntu.com/ubuntu jammy/universe amd64 luarocks all 3.8.0+dfsg1-1 [140 kB] 2026-02-22T21:53:57.140 INFO:teuthology.orchestra.run.trial093.stdout:Get:37 https://archive.ubuntu.com/ubuntu jammy-updates/main amd64 liboath0 amd64 2.6.7-3ubuntu0.1 [41.3 kB] 2026-02-22T21:53:57.140 INFO:teuthology.orchestra.run.trial093.stdout:Get:38 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.functools all 3.4.0-2 [9030 B] 2026-02-22T21:53:57.140 INFO:teuthology.orchestra.run.trial093.stdout:Get:39 https://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-cheroot all 8.5.2+ds1-1ubuntu3.1 [71.1 kB] 2026-02-22T21:53:57.141 INFO:teuthology.orchestra.run.trial093.stdout:Get:40 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.classes all 3.2.1-3 [6452 B] 2026-02-22T21:53:57.141 INFO:teuthology.orchestra.run.trial093.stdout:Get:41 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.text all 3.6.0-2 [8716 B] 2026-02-22T21:53:57.212 INFO:teuthology.orchestra.run.trial093.stdout:Get:42 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.collections all 3.4.0-2 [11.4 kB] 2026-02-22T21:53:57.212 INFO:teuthology.orchestra.run.trial093.stdout:Get:43 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempora all 4.1.2-1 [14.8 kB] 2026-02-22T21:53:57.212 INFO:teuthology.orchestra.run.trial093.stdout:Get:44 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-portend all 3.0.0-1 [7240 B] 2026-02-22T21:53:57.213 INFO:teuthology.orchestra.run.trial093.stdout:Get:45 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-zc.lockfile all 2.0-1 [8980 B] 2026-02-22T21:53:57.284 INFO:teuthology.orchestra.run.trial093.stdout:Get:46 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cherrypy3 all 18.6.1-4 [208 kB] 2026-02-22T21:53:57.286 INFO:teuthology.orchestra.run.trial093.stdout:Get:47 https://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-natsort all 8.0.2-1 [35.3 kB] 2026-02-22T21:53:57.286 INFO:teuthology.orchestra.run.trial093.stdout:Get:48 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-logutils all 0.3.3-8 [17.6 kB] 2026-02-22T21:53:57.287 INFO:teuthology.orchestra.run.trial093.stdout:Get:49 https://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-mako all 1.1.3+ds1-2ubuntu0.1 [60.5 kB] 2026-02-22T21:53:57.287 INFO:teuthology.orchestra.run.trial093.stdout:Get:50 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2026-02-22T21:53:57.287 INFO:teuthology.orchestra.run.trial093.stdout:Get:51 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-singledispatch all 3.4.0.3-3 [7320 B] 2026-02-22T21:53:57.356 INFO:teuthology.orchestra.run.trial093.stdout:Get:52 https://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-webob all 1:1.8.6-1.1ubuntu0.1 [86.7 kB] 2026-02-22T21:53:57.357 INFO:teuthology.orchestra.run.trial093.stdout:Get:53 https://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-waitress all 1.4.4-1.1ubuntu1.1 [47.0 kB] 2026-02-22T21:53:57.357 INFO:teuthology.orchestra.run.trial093.stdout:Get:54 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempita all 0.5.2-6ubuntu1 [15.1 kB] 2026-02-22T21:53:57.358 INFO:teuthology.orchestra.run.trial093.stdout:Get:55 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-paste all 3.5.0+dfsg1-1 [456 kB] 2026-02-22T21:53:57.424 INFO:teuthology.orchestra.run.trial093.stdout:Get:56 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2026-02-22-1629-squid/f90f4eaebb28b619cf76e80cbbe997656da69579/ubuntu/jammy/flavors/default jammy/main amd64 ceph amd64 19.2.3-689-gf90f4eae-1jammy [14.2 kB] 2026-02-22T21:53:57.424 INFO:teuthology.orchestra.run.trial093.stdout:Get:57 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2026-02-22-1629-squid/f90f4eaebb28b619cf76e80cbbe997656da69579/ubuntu/jammy/flavors/default jammy/main amd64 ceph-fuse amd64 19.2.3-689-gf90f4eae-1jammy [1173 kB] 2026-02-22T21:53:57.428 INFO:teuthology.orchestra.run.trial093.stdout:Get:58 https://archive.ubuntu.com/ubuntu jammy/main amd64 python-pastedeploy-tpl all 2.1.1-1 [4892 B] 2026-02-22T21:53:57.428 INFO:teuthology.orchestra.run.trial093.stdout:Get:59 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastedeploy all 2.1.1-1 [26.6 kB] 2026-02-22T21:53:57.428 INFO:teuthology.orchestra.run.trial093.stdout:Get:60 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webtest all 2.0.35-1 [28.5 kB] 2026-02-22T21:53:57.429 INFO:teuthology.orchestra.run.trial093.stdout:Get:61 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pecan all 1.3.3-4ubuntu2 [87.3 kB] 2026-02-22T21:53:57.430 INFO:teuthology.orchestra.run.trial093.stdout:Get:62 https://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-werkzeug all 2.0.2+dfsg1-1ubuntu0.22.04.3 [181 kB] 2026-02-22T21:53:57.432 INFO:teuthology.orchestra.run.trial093.stdout:Get:63 https://archive.ubuntu.com/ubuntu jammy/universe amd64 libfuse2 amd64 2.9.9-5ubuntu3 [90.3 kB] 2026-02-22T21:53:57.440 INFO:teuthology.orchestra.run.trial093.stdout:Get:64 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2026-02-22-1629-squid/f90f4eaebb28b619cf76e80cbbe997656da69579/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mds amd64 19.2.3-689-gf90f4eae-1jammy [2503 kB] 2026-02-22T21:53:57.482 INFO:teuthology.orchestra.run.trial093.stdout:Get:65 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2026-02-22-1629-squid/f90f4eaebb28b619cf76e80cbbe997656da69579/ubuntu/jammy/flavors/default jammy/main amd64 cephadm amd64 19.2.3-689-gf90f4eae-1jammy [798 kB] 2026-02-22T21:53:57.494 INFO:teuthology.orchestra.run.trial093.stdout:Get:66 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2026-02-22-1629-squid/f90f4eaebb28b619cf76e80cbbe997656da69579/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-cephadm all 19.2.3-689-gf90f4eae-1jammy [157 kB] 2026-02-22T21:53:57.497 INFO:teuthology.orchestra.run.trial093.stdout:Get:67 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2026-02-22-1629-squid/f90f4eaebb28b619cf76e80cbbe997656da69579/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-dashboard all 19.2.3-689-gf90f4eae-1jammy [2396 kB] 2026-02-22T21:53:57.500 INFO:teuthology.orchestra.run.trial093.stdout:Get:68 https://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 python3-asyncssh all 2.5.0-1ubuntu0.1 [189 kB] 2026-02-22T21:53:57.503 INFO:teuthology.orchestra.run.trial093.stdout:Get:69 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2026-02-22T21:53:57.503 INFO:teuthology.orchestra.run.trial093.stdout:Get:70 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-routes all 2.5.1-1ubuntu1 [89.0 kB] 2026-02-22T21:53:57.503 INFO:teuthology.orchestra.run.trial093.stdout:Get:71 https://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn-lib amd64 0.23.2-5ubuntu6 [2058 kB] 2026-02-22T21:53:57.537 INFO:teuthology.orchestra.run.trial093.stdout:Get:72 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2026-02-22-1629-squid/f90f4eaebb28b619cf76e80cbbe997656da69579/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-diskprediction-local all 19.2.3-689-gf90f4eae-1jammy [8625 kB] 2026-02-22T21:53:57.573 INFO:teuthology.orchestra.run.trial093.stdout:Get:73 https://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-joblib all 0.17.0-4ubuntu1 [204 kB] 2026-02-22T21:53:57.575 INFO:teuthology.orchestra.run.trial093.stdout:Get:74 https://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-threadpoolctl all 3.1.0-1 [21.3 kB] 2026-02-22T21:53:57.575 INFO:teuthology.orchestra.run.trial093.stdout:Get:75 https://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn all 0.23.2-5ubuntu6 [1829 kB] 2026-02-22T21:53:57.596 INFO:teuthology.orchestra.run.trial093.stdout:Get:76 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cachetools all 5.0.0-1 [9722 B] 2026-02-22T21:53:57.596 INFO:teuthology.orchestra.run.trial093.stdout:Get:77 https://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-rsa all 4.8-1 [28.4 kB] 2026-02-22T21:53:57.596 INFO:teuthology.orchestra.run.trial093.stdout:Get:78 https://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-google-auth all 1.5.1-3 [35.7 kB] 2026-02-22T21:53:57.645 INFO:teuthology.orchestra.run.trial093.stdout:Get:79 https://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-requests-oauthlib all 1.3.0+ds-0.1 [18.7 kB] 2026-02-22T21:53:57.645 INFO:teuthology.orchestra.run.trial093.stdout:Get:80 https://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-websocket all 1.2.3-1 [34.7 kB] 2026-02-22T21:53:57.646 INFO:teuthology.orchestra.run.trial093.stdout:Get:81 https://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-kubernetes all 12.0.1-1ubuntu1 [353 kB] 2026-02-22T21:53:57.649 INFO:teuthology.orchestra.run.trial093.stdout:Get:82 https://archive.ubuntu.com/ubuntu jammy/main amd64 libonig5 amd64 6.9.7.1-2build1 [172 kB] 2026-02-22T21:53:57.679 INFO:teuthology.orchestra.run.trial093.stdout:Get:83 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2026-02-22-1629-squid/f90f4eaebb28b619cf76e80cbbe997656da69579/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-k8sevents all 19.2.3-689-gf90f4eae-1jammy [14.3 kB] 2026-02-22T21:53:57.680 INFO:teuthology.orchestra.run.trial093.stdout:Get:84 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2026-02-22-1629-squid/f90f4eaebb28b619cf76e80cbbe997656da69579/ubuntu/jammy/flavors/default jammy/main amd64 ceph-test amd64 19.2.3-689-gf90f4eae-1jammy [52.1 MB] 2026-02-22T21:53:57.717 INFO:teuthology.orchestra.run.trial093.stdout:Get:85 https://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libjq1 amd64 1.6-2.1ubuntu3.1 [133 kB] 2026-02-22T21:53:57.718 INFO:teuthology.orchestra.run.trial093.stdout:Get:86 https://archive.ubuntu.com/ubuntu jammy-updates/main amd64 jq amd64 1.6-2.1ubuntu3.1 [52.5 kB] 2026-02-22T21:53:57.719 INFO:teuthology.orchestra.run.trial093.stdout:Get:87 https://archive.ubuntu.com/ubuntu jammy/main amd64 socat amd64 1.7.4.1-3ubuntu4 [349 kB] 2026-02-22T21:53:57.722 INFO:teuthology.orchestra.run.trial093.stdout:Get:88 https://archive.ubuntu.com/ubuntu jammy/universe amd64 xmlstarlet amd64 1.6.1-2.1 [265 kB] 2026-02-22T21:53:57.725 INFO:teuthology.orchestra.run.trial093.stdout:Get:89 https://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-socket amd64 3.0~rc1+git+ac3201d-6 [78.9 kB] 2026-02-22T21:53:57.726 INFO:teuthology.orchestra.run.trial093.stdout:Get:90 https://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-sec amd64 1.0.2-1 [37.6 kB] 2026-02-22T21:53:57.789 INFO:teuthology.orchestra.run.trial093.stdout:Get:91 https://archive.ubuntu.com/ubuntu jammy-updates/main amd64 nvme-cli amd64 1.16-3ubuntu0.3 [474 kB] 2026-02-22T21:53:57.794 INFO:teuthology.orchestra.run.trial093.stdout:Get:92 https://archive.ubuntu.com/ubuntu jammy/main amd64 pkg-config amd64 0.29.2-1ubuntu3 [48.2 kB] 2026-02-22T21:53:57.795 INFO:teuthology.orchestra.run.trial093.stdout:Get:93 https://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 python-asyncssh-doc all 2.5.0-1ubuntu0.1 [309 kB] 2026-02-22T21:53:57.797 INFO:teuthology.orchestra.run.trial093.stdout:Get:94 https://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-iniconfig all 1.1.1-2 [6024 B] 2026-02-22T21:53:57.861 INFO:teuthology.orchestra.run.trial093.stdout:Get:95 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2026-02-22T21:53:57.862 INFO:teuthology.orchestra.run.trial093.stdout:Get:96 https://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pluggy all 0.13.0-7.1 [19.0 kB] 2026-02-22T21:53:57.862 INFO:teuthology.orchestra.run.trial093.stdout:Get:97 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-psutil amd64 5.9.0-1build1 [158 kB] 2026-02-22T21:53:57.864 INFO:teuthology.orchestra.run.trial093.stdout:Get:98 https://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-py all 1.10.0-1 [71.9 kB] 2026-02-22T21:53:57.864 INFO:teuthology.orchestra.run.trial093.stdout:Get:99 https://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-pygments all 2.11.2+dfsg-2ubuntu0.1 [750 kB] 2026-02-22T21:53:57.872 INFO:teuthology.orchestra.run.trial093.stdout:Get:100 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyinotify all 0.9.6-1.3 [24.8 kB] 2026-02-22T21:53:57.934 INFO:teuthology.orchestra.run.trial093.stdout:Get:101 https://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-toml all 0.10.2-1 [16.5 kB] 2026-02-22T21:53:57.934 INFO:teuthology.orchestra.run.trial093.stdout:Get:102 https://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pytest all 6.2.5-1ubuntu2 [214 kB] 2026-02-22T21:53:57.936 INFO:teuthology.orchestra.run.trial093.stdout:Get:103 https://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplejson amd64 3.17.6-1build1 [54.7 kB] 2026-02-22T21:53:57.936 INFO:teuthology.orchestra.run.trial093.stdout:Get:104 https://archive.ubuntu.com/ubuntu jammy/universe amd64 qttranslations5-l10n all 5.15.3-1 [1983 kB] 2026-02-22T21:53:58.539 INFO:teuthology.orchestra.run.trial093.stdout:Get:105 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2026-02-22-1629-squid/f90f4eaebb28b619cf76e80cbbe997656da69579/ubuntu/jammy/flavors/default jammy/main amd64 ceph-volume all 19.2.3-689-gf90f4eae-1jammy [135 kB] 2026-02-22T21:53:58.540 INFO:teuthology.orchestra.run.trial093.stdout:Get:106 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2026-02-22-1629-squid/f90f4eaebb28b619cf76e80cbbe997656da69579/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs-dev amd64 19.2.3-689-gf90f4eae-1jammy [41.0 kB] 2026-02-22T21:53:58.540 INFO:teuthology.orchestra.run.trial093.stdout:Get:107 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2026-02-22-1629-squid/f90f4eaebb28b619cf76e80cbbe997656da69579/ubuntu/jammy/flavors/default jammy/main amd64 radosgw amd64 19.2.3-689-gf90f4eae-1jammy [13.7 MB] 2026-02-22T21:53:58.761 INFO:teuthology.orchestra.run.trial093.stdout:Get:108 https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2026-02-22-1629-squid/f90f4eaebb28b619cf76e80cbbe997656da69579/ubuntu/jammy/flavors/default jammy/main amd64 rbd-fuse amd64 19.2.3-689-gf90f4eae-1jammy [92.3 kB] 2026-02-22T21:53:59.083 INFO:teuthology.orchestra.run.trial093.stdout:Fetched 178 MB in 3s (64.8 MB/s) 2026-02-22T21:53:59.103 INFO:teuthology.orchestra.run.trial093.stdout:Selecting previously unselected package liblttng-ust1:amd64. 2026-02-22T21:53:59.263 INFO:teuthology.orchestra.run.trial093.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 167442 files and directories currently installed.) 2026-02-22T21:53:59.265 INFO:teuthology.orchestra.run.trial093.stdout:Preparing to unpack .../000-liblttng-ust1_2.13.1-1ubuntu1_amd64.deb ... 2026-02-22T21:53:59.266 INFO:teuthology.orchestra.run.trial093.stdout:Unpacking liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2026-02-22T21:53:59.288 INFO:teuthology.orchestra.run.trial093.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2026-02-22T21:53:59.301 INFO:teuthology.orchestra.run.trial093.stdout:Preparing to unpack .../001-libdouble-conversion3_3.1.7-4_amd64.deb ... 2026-02-22T21:53:59.301 INFO:teuthology.orchestra.run.trial093.stdout:Unpacking libdouble-conversion3:amd64 (3.1.7-4) ... 2026-02-22T21:53:59.317 INFO:teuthology.orchestra.run.trial093.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2026-02-22T21:53:59.328 INFO:teuthology.orchestra.run.trial093.stdout:Preparing to unpack .../002-libpcre2-16-0_10.39-3ubuntu0.1_amd64.deb ... 2026-02-22T21:53:59.329 INFO:teuthology.orchestra.run.trial093.stdout:Unpacking libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2026-02-22T21:53:59.348 INFO:teuthology.orchestra.run.trial093.stdout:Selecting previously unselected package libqt5core5a:amd64. 2026-02-22T21:53:59.357 INFO:teuthology.orchestra.run.trial093.stdout:Preparing to unpack .../003-libqt5core5a_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2026-02-22T21:53:59.362 INFO:teuthology.orchestra.run.trial093.stdout:Unpacking libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2026-02-22T21:53:59.399 INFO:teuthology.orchestra.run.trial093.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2026-02-22T21:53:59.408 INFO:teuthology.orchestra.run.trial093.stdout:Preparing to unpack .../004-libqt5dbus5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2026-02-22T21:53:59.409 INFO:teuthology.orchestra.run.trial093.stdout:Unpacking libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2026-02-22T21:53:59.425 INFO:teuthology.orchestra.run.trial093.stdout:Selecting previously unselected package libqt5network5:amd64. 2026-02-22T21:53:59.433 INFO:teuthology.orchestra.run.trial093.stdout:Preparing to unpack .../005-libqt5network5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2026-02-22T21:53:59.433 INFO:teuthology.orchestra.run.trial093.stdout:Unpacking libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2026-02-22T21:53:59.453 INFO:teuthology.orchestra.run.trial093.stdout:Selecting previously unselected package libthrift-0.16.0:amd64. 2026-02-22T21:53:59.461 INFO:teuthology.orchestra.run.trial093.stdout:Preparing to unpack .../006-libthrift-0.16.0_0.16.0-2_amd64.deb ... 2026-02-22T21:53:59.462 INFO:teuthology.orchestra.run.trial093.stdout:Unpacking libthrift-0.16.0:amd64 (0.16.0-2) ... 2026-02-22T21:53:59.478 INFO:teuthology.orchestra.run.trial093.stdout:Selecting previously unselected package librados2. 2026-02-22T21:53:59.486 INFO:teuthology.orchestra.run.trial093.stdout:Preparing to unpack .../007-librados2_19.2.3-689-gf90f4eae-1jammy_amd64.deb ... 2026-02-22T21:53:59.486 INFO:teuthology.orchestra.run.trial093.stdout:Unpacking librados2 (19.2.3-689-gf90f4eae-1jammy) ... 2026-02-22T21:53:59.528 INFO:teuthology.orchestra.run.trial093.stdout:Selecting previously unselected package libnbd0. 2026-02-22T21:53:59.536 INFO:teuthology.orchestra.run.trial093.stdout:Preparing to unpack .../008-libnbd0_1.10.5-1_amd64.deb ... 2026-02-22T21:53:59.536 INFO:teuthology.orchestra.run.trial093.stdout:Unpacking libnbd0 (1.10.5-1) ... 2026-02-22T21:53:59.551 INFO:teuthology.orchestra.run.trial093.stdout:Selecting previously unselected package librbd1. 2026-02-22T21:53:59.559 INFO:teuthology.orchestra.run.trial093.stdout:Preparing to unpack .../009-librbd1_19.2.3-689-gf90f4eae-1jammy_amd64.deb ... 2026-02-22T21:53:59.560 INFO:teuthology.orchestra.run.trial093.stdout:Unpacking librbd1 (19.2.3-689-gf90f4eae-1jammy) ... 2026-02-22T21:53:59.601 INFO:teuthology.orchestra.run.trial093.stdout:Selecting previously unselected package libcephfs2. 2026-02-22T21:53:59.609 INFO:teuthology.orchestra.run.trial093.stdout:Preparing to unpack .../010-libcephfs2_19.2.3-689-gf90f4eae-1jammy_amd64.deb ... 2026-02-22T21:53:59.610 INFO:teuthology.orchestra.run.trial093.stdout:Unpacking libcephfs2 (19.2.3-689-gf90f4eae-1jammy) ... 2026-02-22T21:53:59.631 INFO:teuthology.orchestra.run.trial093.stdout:Selecting previously unselected package python3-rados. 2026-02-22T21:53:59.639 INFO:teuthology.orchestra.run.trial093.stdout:Preparing to unpack .../011-python3-rados_19.2.3-689-gf90f4eae-1jammy_amd64.deb ... 2026-02-22T21:53:59.639 INFO:teuthology.orchestra.run.trial093.stdout:Unpacking python3-rados (19.2.3-689-gf90f4eae-1jammy) ... 2026-02-22T21:53:59.656 INFO:teuthology.orchestra.run.trial093.stdout:Selecting previously unselected package python3-ceph-argparse. 2026-02-22T21:53:59.664 INFO:teuthology.orchestra.run.trial093.stdout:Preparing to unpack .../012-python3-ceph-argparse_19.2.3-689-gf90f4eae-1jammy_all.deb ... 2026-02-22T21:53:59.664 INFO:teuthology.orchestra.run.trial093.stdout:Unpacking python3-ceph-argparse (19.2.3-689-gf90f4eae-1jammy) ... 2026-02-22T21:53:59.678 INFO:teuthology.orchestra.run.trial093.stdout:Selecting previously unselected package python3-cephfs. 2026-02-22T21:53:59.686 INFO:teuthology.orchestra.run.trial093.stdout:Preparing to unpack .../013-python3-cephfs_19.2.3-689-gf90f4eae-1jammy_amd64.deb ... 2026-02-22T21:53:59.686 INFO:teuthology.orchestra.run.trial093.stdout:Unpacking python3-cephfs (19.2.3-689-gf90f4eae-1jammy) ... 2026-02-22T21:53:59.701 INFO:teuthology.orchestra.run.trial093.stdout:Selecting previously unselected package python3-ceph-common. 2026-02-22T21:53:59.710 INFO:teuthology.orchestra.run.trial093.stdout:Preparing to unpack .../014-python3-ceph-common_19.2.3-689-gf90f4eae-1jammy_all.deb ... 2026-02-22T21:53:59.710 INFO:teuthology.orchestra.run.trial093.stdout:Unpacking python3-ceph-common (19.2.3-689-gf90f4eae-1jammy) ... 2026-02-22T21:53:59.728 INFO:teuthology.orchestra.run.trial093.stdout:Selecting previously unselected package python3-wcwidth. 2026-02-22T21:53:59.735 INFO:teuthology.orchestra.run.trial093.stdout:Preparing to unpack .../015-python3-wcwidth_0.2.5+dfsg1-1_all.deb ... 2026-02-22T21:53:59.736 INFO:teuthology.orchestra.run.trial093.stdout:Unpacking python3-wcwidth (0.2.5+dfsg1-1) ... 2026-02-22T21:53:59.751 INFO:teuthology.orchestra.run.trial093.stdout:Selecting previously unselected package python3-prettytable. 2026-02-22T21:53:59.760 INFO:teuthology.orchestra.run.trial093.stdout:Preparing to unpack .../016-python3-prettytable_2.5.0-2_all.deb ... 2026-02-22T21:53:59.760 INFO:teuthology.orchestra.run.trial093.stdout:Unpacking python3-prettytable (2.5.0-2) ... 2026-02-22T21:53:59.775 INFO:teuthology.orchestra.run.trial093.stdout:Selecting previously unselected package python3-rbd. 2026-02-22T21:53:59.782 INFO:teuthology.orchestra.run.trial093.stdout:Preparing to unpack .../017-python3-rbd_19.2.3-689-gf90f4eae-1jammy_amd64.deb ... 2026-02-22T21:53:59.783 INFO:teuthology.orchestra.run.trial093.stdout:Unpacking python3-rbd (19.2.3-689-gf90f4eae-1jammy) ... 2026-02-22T21:53:59.800 INFO:teuthology.orchestra.run.trial093.stdout:Selecting previously unselected package librdkafka1:amd64. 2026-02-22T21:53:59.808 INFO:teuthology.orchestra.run.trial093.stdout:Preparing to unpack .../018-librdkafka1_1.8.0-1build1_amd64.deb ... 2026-02-22T21:53:59.808 INFO:teuthology.orchestra.run.trial093.stdout:Unpacking librdkafka1:amd64 (1.8.0-1build1) ... 2026-02-22T21:53:59.827 INFO:teuthology.orchestra.run.trial093.stdout:Selecting previously unselected package libreadline-dev:amd64. 2026-02-22T21:53:59.836 INFO:teuthology.orchestra.run.trial093.stdout:Preparing to unpack .../019-libreadline-dev_8.1.2-1_amd64.deb ... 2026-02-22T21:53:59.836 INFO:teuthology.orchestra.run.trial093.stdout:Unpacking libreadline-dev:amd64 (8.1.2-1) ... 2026-02-22T21:53:59.854 INFO:teuthology.orchestra.run.trial093.stdout:Selecting previously unselected package liblua5.3-dev:amd64. 2026-02-22T21:53:59.862 INFO:teuthology.orchestra.run.trial093.stdout:Preparing to unpack .../020-liblua5.3-dev_5.3.6-1build1_amd64.deb ... 2026-02-22T21:53:59.862 INFO:teuthology.orchestra.run.trial093.stdout:Unpacking liblua5.3-dev:amd64 (5.3.6-1build1) ... 2026-02-22T21:53:59.879 INFO:teuthology.orchestra.run.trial093.stdout:Selecting previously unselected package lua5.1. 2026-02-22T21:53:59.887 INFO:teuthology.orchestra.run.trial093.stdout:Preparing to unpack .../021-lua5.1_5.1.5-8.1build4_amd64.deb ... 2026-02-22T21:53:59.887 INFO:teuthology.orchestra.run.trial093.stdout:Unpacking lua5.1 (5.1.5-8.1build4) ... 2026-02-22T21:53:59.906 INFO:teuthology.orchestra.run.trial093.stdout:Selecting previously unselected package lua-any. 2026-02-22T21:53:59.914 INFO:teuthology.orchestra.run.trial093.stdout:Preparing to unpack .../022-lua-any_27ubuntu1_all.deb ... 2026-02-22T21:53:59.915 INFO:teuthology.orchestra.run.trial093.stdout:Unpacking lua-any (27ubuntu1) ... 2026-02-22T21:53:59.929 INFO:teuthology.orchestra.run.trial093.stdout:Selecting previously unselected package zip. 2026-02-22T21:53:59.937 INFO:teuthology.orchestra.run.trial093.stdout:Preparing to unpack .../023-zip_3.0-12build2_amd64.deb ... 2026-02-22T21:53:59.937 INFO:teuthology.orchestra.run.trial093.stdout:Unpacking zip (3.0-12build2) ... 2026-02-22T21:53:59.956 INFO:teuthology.orchestra.run.trial093.stdout:Selecting previously unselected package unzip. 2026-02-22T21:53:59.964 INFO:teuthology.orchestra.run.trial093.stdout:Preparing to unpack .../024-unzip_6.0-26ubuntu3.2_amd64.deb ... 2026-02-22T21:53:59.964 INFO:teuthology.orchestra.run.trial093.stdout:Unpacking unzip (6.0-26ubuntu3.2) ... 2026-02-22T21:53:59.982 INFO:teuthology.orchestra.run.trial093.stdout:Selecting previously unselected package luarocks. 2026-02-22T21:53:59.990 INFO:teuthology.orchestra.run.trial093.stdout:Preparing to unpack .../025-luarocks_3.8.0+dfsg1-1_all.deb ... 2026-02-22T21:53:59.991 INFO:teuthology.orchestra.run.trial093.stdout:Unpacking luarocks (3.8.0+dfsg1-1) ... 2026-02-22T21:54:00.029 INFO:teuthology.orchestra.run.trial093.stdout:Selecting previously unselected package librgw2. 2026-02-22T21:54:00.037 INFO:teuthology.orchestra.run.trial093.stdout:Preparing to unpack .../026-librgw2_19.2.3-689-gf90f4eae-1jammy_amd64.deb ... 2026-02-22T21:54:00.037 INFO:teuthology.orchestra.run.trial093.stdout:Unpacking librgw2 (19.2.3-689-gf90f4eae-1jammy) ... 2026-02-22T21:54:00.109 INFO:teuthology.orchestra.run.trial093.stdout:Selecting previously unselected package python3-rgw. 2026-02-22T21:54:00.117 INFO:teuthology.orchestra.run.trial093.stdout:Preparing to unpack .../027-python3-rgw_19.2.3-689-gf90f4eae-1jammy_amd64.deb ... 2026-02-22T21:54:00.118 INFO:teuthology.orchestra.run.trial093.stdout:Unpacking python3-rgw (19.2.3-689-gf90f4eae-1jammy) ... 2026-02-22T21:54:00.133 INFO:teuthology.orchestra.run.trial093.stdout:Selecting previously unselected package liboath0:amd64. 2026-02-22T21:54:00.141 INFO:teuthology.orchestra.run.trial093.stdout:Preparing to unpack .../028-liboath0_2.6.7-3ubuntu0.1_amd64.deb ... 2026-02-22T21:54:00.142 INFO:teuthology.orchestra.run.trial093.stdout:Unpacking liboath0:amd64 (2.6.7-3ubuntu0.1) ... 2026-02-22T21:54:00.157 INFO:teuthology.orchestra.run.trial093.stdout:Selecting previously unselected package libradosstriper1. 2026-02-22T21:54:00.165 INFO:teuthology.orchestra.run.trial093.stdout:Preparing to unpack .../029-libradosstriper1_19.2.3-689-gf90f4eae-1jammy_amd64.deb ... 2026-02-22T21:54:00.165 INFO:teuthology.orchestra.run.trial093.stdout:Unpacking libradosstriper1 (19.2.3-689-gf90f4eae-1jammy) ... 2026-02-22T21:54:00.184 INFO:teuthology.orchestra.run.trial093.stdout:Selecting previously unselected package ceph-common. 2026-02-22T21:54:00.192 INFO:teuthology.orchestra.run.trial093.stdout:Preparing to unpack .../030-ceph-common_19.2.3-689-gf90f4eae-1jammy_amd64.deb ... 2026-02-22T21:54:00.192 INFO:teuthology.orchestra.run.trial093.stdout:Unpacking ceph-common (19.2.3-689-gf90f4eae-1jammy) ... 2026-02-22T21:54:00.394 INFO:teuthology.orchestra.run.trial093.stdout:Selecting previously unselected package ceph-base. 2026-02-22T21:54:00.402 INFO:teuthology.orchestra.run.trial093.stdout:Preparing to unpack .../031-ceph-base_19.2.3-689-gf90f4eae-1jammy_amd64.deb ... 2026-02-22T21:54:00.407 INFO:teuthology.orchestra.run.trial093.stdout:Unpacking ceph-base (19.2.3-689-gf90f4eae-1jammy) ... 2026-02-22T21:54:00.472 INFO:teuthology.orchestra.run.trial093.stdout:Selecting previously unselected package python3-jaraco.functools. 2026-02-22T21:54:00.481 INFO:teuthology.orchestra.run.trial093.stdout:Preparing to unpack .../032-python3-jaraco.functools_3.4.0-2_all.deb ... 2026-02-22T21:54:00.481 INFO:teuthology.orchestra.run.trial093.stdout:Unpacking python3-jaraco.functools (3.4.0-2) ... 2026-02-22T21:54:00.496 INFO:teuthology.orchestra.run.trial093.stdout:Selecting previously unselected package python3-cheroot. 2026-02-22T21:54:00.504 INFO:teuthology.orchestra.run.trial093.stdout:Preparing to unpack .../033-python3-cheroot_8.5.2+ds1-1ubuntu3.1_all.deb ... 2026-02-22T21:54:00.505 INFO:teuthology.orchestra.run.trial093.stdout:Unpacking python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2026-02-22T21:54:00.522 INFO:teuthology.orchestra.run.trial093.stdout:Selecting previously unselected package python3-jaraco.classes. 2026-02-22T21:54:00.530 INFO:teuthology.orchestra.run.trial093.stdout:Preparing to unpack .../034-python3-jaraco.classes_3.2.1-3_all.deb ... 2026-02-22T21:54:00.530 INFO:teuthology.orchestra.run.trial093.stdout:Unpacking python3-jaraco.classes (3.2.1-3) ... 2026-02-22T21:54:00.545 INFO:teuthology.orchestra.run.trial093.stdout:Selecting previously unselected package python3-jaraco.text. 2026-02-22T21:54:00.553 INFO:teuthology.orchestra.run.trial093.stdout:Preparing to unpack .../035-python3-jaraco.text_3.6.0-2_all.deb ... 2026-02-22T21:54:00.554 INFO:teuthology.orchestra.run.trial093.stdout:Unpacking python3-jaraco.text (3.6.0-2) ... 2026-02-22T21:54:00.569 INFO:teuthology.orchestra.run.trial093.stdout:Selecting previously unselected package python3-jaraco.collections. 2026-02-22T21:54:00.577 INFO:teuthology.orchestra.run.trial093.stdout:Preparing to unpack .../036-python3-jaraco.collections_3.4.0-2_all.deb ... 2026-02-22T21:54:00.577 INFO:teuthology.orchestra.run.trial093.stdout:Unpacking python3-jaraco.collections (3.4.0-2) ... 2026-02-22T21:54:00.592 INFO:teuthology.orchestra.run.trial093.stdout:Selecting previously unselected package python3-tempora. 2026-02-22T21:54:00.601 INFO:teuthology.orchestra.run.trial093.stdout:Preparing to unpack .../037-python3-tempora_4.1.2-1_all.deb ... 2026-02-22T21:54:00.601 INFO:teuthology.orchestra.run.trial093.stdout:Unpacking python3-tempora (4.1.2-1) ... 2026-02-22T21:54:00.616 INFO:teuthology.orchestra.run.trial093.stdout:Selecting previously unselected package python3-portend. 2026-02-22T21:54:00.624 INFO:teuthology.orchestra.run.trial093.stdout:Preparing to unpack .../038-python3-portend_3.0.0-1_all.deb ... 2026-02-22T21:54:00.624 INFO:teuthology.orchestra.run.trial093.stdout:Unpacking python3-portend (3.0.0-1) ... 2026-02-22T21:54:00.639 INFO:teuthology.orchestra.run.trial093.stdout:Selecting previously unselected package python3-zc.lockfile. 2026-02-22T21:54:00.648 INFO:teuthology.orchestra.run.trial093.stdout:Preparing to unpack .../039-python3-zc.lockfile_2.0-1_all.deb ... 2026-02-22T21:54:00.648 INFO:teuthology.orchestra.run.trial093.stdout:Unpacking python3-zc.lockfile (2.0-1) ... 2026-02-22T21:54:00.662 INFO:teuthology.orchestra.run.trial093.stdout:Selecting previously unselected package python3-cherrypy3. 2026-02-22T21:54:00.670 INFO:teuthology.orchestra.run.trial093.stdout:Preparing to unpack .../040-python3-cherrypy3_18.6.1-4_all.deb ... 2026-02-22T21:54:00.671 INFO:teuthology.orchestra.run.trial093.stdout:Unpacking python3-cherrypy3 (18.6.1-4) ... 2026-02-22T21:54:00.694 INFO:teuthology.orchestra.run.trial093.stdout:Selecting previously unselected package python3-natsort. 2026-02-22T21:54:00.702 INFO:teuthology.orchestra.run.trial093.stdout:Preparing to unpack .../041-python3-natsort_8.0.2-1_all.deb ... 2026-02-22T21:54:00.703 INFO:teuthology.orchestra.run.trial093.stdout:Unpacking python3-natsort (8.0.2-1) ... 2026-02-22T21:54:00.719 INFO:teuthology.orchestra.run.trial093.stdout:Selecting previously unselected package python3-logutils. 2026-02-22T21:54:00.727 INFO:teuthology.orchestra.run.trial093.stdout:Preparing to unpack .../042-python3-logutils_0.3.3-8_all.deb ... 2026-02-22T21:54:00.727 INFO:teuthology.orchestra.run.trial093.stdout:Unpacking python3-logutils (0.3.3-8) ... 2026-02-22T21:54:00.741 INFO:teuthology.orchestra.run.trial093.stdout:Selecting previously unselected package python3-mako. 2026-02-22T21:54:00.749 INFO:teuthology.orchestra.run.trial093.stdout:Preparing to unpack .../043-python3-mako_1.1.3+ds1-2ubuntu0.1_all.deb ... 2026-02-22T21:54:00.750 INFO:teuthology.orchestra.run.trial093.stdout:Unpacking python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2026-02-22T21:54:00.767 INFO:teuthology.orchestra.run.trial093.stdout:Selecting previously unselected package python3-simplegeneric. 2026-02-22T21:54:00.775 INFO:teuthology.orchestra.run.trial093.stdout:Preparing to unpack .../044-python3-simplegeneric_0.8.1-3_all.deb ... 2026-02-22T21:54:00.775 INFO:teuthology.orchestra.run.trial093.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2026-02-22T21:54:00.789 INFO:teuthology.orchestra.run.trial093.stdout:Selecting previously unselected package python3-singledispatch. 2026-02-22T21:54:00.797 INFO:teuthology.orchestra.run.trial093.stdout:Preparing to unpack .../045-python3-singledispatch_3.4.0.3-3_all.deb ... 2026-02-22T21:54:00.797 INFO:teuthology.orchestra.run.trial093.stdout:Unpacking python3-singledispatch (3.4.0.3-3) ... 2026-02-22T21:54:00.812 INFO:teuthology.orchestra.run.trial093.stdout:Selecting previously unselected package python3-webob. 2026-02-22T21:54:00.820 INFO:teuthology.orchestra.run.trial093.stdout:Preparing to unpack .../046-python3-webob_1%3a1.8.6-1.1ubuntu0.1_all.deb ... 2026-02-22T21:54:00.820 INFO:teuthology.orchestra.run.trial093.stdout:Unpacking python3-webob (1:1.8.6-1.1ubuntu0.1) ... 2026-02-22T21:54:00.837 INFO:teuthology.orchestra.run.trial093.stdout:Selecting previously unselected package python3-waitress. 2026-02-22T21:54:00.845 INFO:teuthology.orchestra.run.trial093.stdout:Preparing to unpack .../047-python3-waitress_1.4.4-1.1ubuntu1.1_all.deb ... 2026-02-22T21:54:00.847 INFO:teuthology.orchestra.run.trial093.stdout:Unpacking python3-waitress (1.4.4-1.1ubuntu1.1) ... 2026-02-22T21:54:00.863 INFO:teuthology.orchestra.run.trial093.stdout:Selecting previously unselected package python3-tempita. 2026-02-22T21:54:00.871 INFO:teuthology.orchestra.run.trial093.stdout:Preparing to unpack .../048-python3-tempita_0.5.2-6ubuntu1_all.deb ... 2026-02-22T21:54:00.871 INFO:teuthology.orchestra.run.trial093.stdout:Unpacking python3-tempita (0.5.2-6ubuntu1) ... 2026-02-22T21:54:00.885 INFO:teuthology.orchestra.run.trial093.stdout:Selecting previously unselected package python3-paste. 2026-02-22T21:54:00.893 INFO:teuthology.orchestra.run.trial093.stdout:Preparing to unpack .../049-python3-paste_3.5.0+dfsg1-1_all.deb ... 2026-02-22T21:54:00.893 INFO:teuthology.orchestra.run.trial093.stdout:Unpacking python3-paste (3.5.0+dfsg1-1) ... 2026-02-22T21:54:00.929 INFO:teuthology.orchestra.run.trial093.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2026-02-22T21:54:00.936 INFO:teuthology.orchestra.run.trial093.stdout:Preparing to unpack .../050-python-pastedeploy-tpl_2.1.1-1_all.deb ... 2026-02-22T21:54:00.937 INFO:teuthology.orchestra.run.trial093.stdout:Unpacking python-pastedeploy-tpl (2.1.1-1) ... 2026-02-22T21:54:00.951 INFO:teuthology.orchestra.run.trial093.stdout:Selecting previously unselected package python3-pastedeploy. 2026-02-22T21:54:00.959 INFO:teuthology.orchestra.run.trial093.stdout:Preparing to unpack .../051-python3-pastedeploy_2.1.1-1_all.deb ... 2026-02-22T21:54:00.959 INFO:teuthology.orchestra.run.trial093.stdout:Unpacking python3-pastedeploy (2.1.1-1) ... 2026-02-22T21:54:00.975 INFO:teuthology.orchestra.run.trial093.stdout:Selecting previously unselected package python3-webtest. 2026-02-22T21:54:00.983 INFO:teuthology.orchestra.run.trial093.stdout:Preparing to unpack .../052-python3-webtest_2.0.35-1_all.deb ... 2026-02-22T21:54:00.983 INFO:teuthology.orchestra.run.trial093.stdout:Unpacking python3-webtest (2.0.35-1) ... 2026-02-22T21:54:00.999 INFO:teuthology.orchestra.run.trial093.stdout:Selecting previously unselected package python3-pecan. 2026-02-22T21:54:01.007 INFO:teuthology.orchestra.run.trial093.stdout:Preparing to unpack .../053-python3-pecan_1.3.3-4ubuntu2_all.deb ... 2026-02-22T21:54:01.008 INFO:teuthology.orchestra.run.trial093.stdout:Unpacking python3-pecan (1.3.3-4ubuntu2) ... 2026-02-22T21:54:01.032 INFO:teuthology.orchestra.run.trial093.stdout:Selecting previously unselected package python3-werkzeug. 2026-02-22T21:54:01.040 INFO:teuthology.orchestra.run.trial093.stdout:Preparing to unpack .../054-python3-werkzeug_2.0.2+dfsg1-1ubuntu0.22.04.3_all.deb ... 2026-02-22T21:54:01.040 INFO:teuthology.orchestra.run.trial093.stdout:Unpacking python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.3) ... 2026-02-22T21:54:01.060 INFO:teuthology.orchestra.run.trial093.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2026-02-22T21:54:01.068 INFO:teuthology.orchestra.run.trial093.stdout:Preparing to unpack .../055-ceph-mgr-modules-core_19.2.3-689-gf90f4eae-1jammy_all.deb ... 2026-02-22T21:54:01.069 INFO:teuthology.orchestra.run.trial093.stdout:Unpacking ceph-mgr-modules-core (19.2.3-689-gf90f4eae-1jammy) ... 2026-02-22T21:54:01.097 INFO:teuthology.orchestra.run.trial093.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2026-02-22T21:54:01.105 INFO:teuthology.orchestra.run.trial093.stdout:Preparing to unpack .../056-libsqlite3-mod-ceph_19.2.3-689-gf90f4eae-1jammy_amd64.deb ... 2026-02-22T21:54:01.106 INFO:teuthology.orchestra.run.trial093.stdout:Unpacking libsqlite3-mod-ceph (19.2.3-689-gf90f4eae-1jammy) ... 2026-02-22T21:54:01.120 INFO:teuthology.orchestra.run.trial093.stdout:Selecting previously unselected package ceph-mgr. 2026-02-22T21:54:01.128 INFO:teuthology.orchestra.run.trial093.stdout:Preparing to unpack .../057-ceph-mgr_19.2.3-689-gf90f4eae-1jammy_amd64.deb ... 2026-02-22T21:54:01.129 INFO:teuthology.orchestra.run.trial093.stdout:Unpacking ceph-mgr (19.2.3-689-gf90f4eae-1jammy) ... 2026-02-22T21:54:01.151 INFO:teuthology.orchestra.run.trial093.stdout:Selecting previously unselected package ceph-mon. 2026-02-22T21:54:01.159 INFO:teuthology.orchestra.run.trial093.stdout:Preparing to unpack .../058-ceph-mon_19.2.3-689-gf90f4eae-1jammy_amd64.deb ... 2026-02-22T21:54:01.160 INFO:teuthology.orchestra.run.trial093.stdout:Unpacking ceph-mon (19.2.3-689-gf90f4eae-1jammy) ... 2026-02-22T21:54:01.217 INFO:teuthology.orchestra.run.trial093.stdout:Selecting previously unselected package libfuse2:amd64. 2026-02-22T21:54:01.225 INFO:teuthology.orchestra.run.trial093.stdout:Preparing to unpack .../059-libfuse2_2.9.9-5ubuntu3_amd64.deb ... 2026-02-22T21:54:01.226 INFO:teuthology.orchestra.run.trial093.stdout:Unpacking libfuse2:amd64 (2.9.9-5ubuntu3) ... 2026-02-22T21:54:01.242 INFO:teuthology.orchestra.run.trial093.stdout:Selecting previously unselected package ceph-osd. 2026-02-22T21:54:01.250 INFO:teuthology.orchestra.run.trial093.stdout:Preparing to unpack .../060-ceph-osd_19.2.3-689-gf90f4eae-1jammy_amd64.deb ... 2026-02-22T21:54:01.250 INFO:teuthology.orchestra.run.trial093.stdout:Unpacking ceph-osd (19.2.3-689-gf90f4eae-1jammy) ... 2026-02-22T21:54:01.402 INFO:teuthology.orchestra.run.trial093.stdout:Selecting previously unselected package ceph. 2026-02-22T21:54:01.410 INFO:teuthology.orchestra.run.trial093.stdout:Preparing to unpack .../061-ceph_19.2.3-689-gf90f4eae-1jammy_amd64.deb ... 2026-02-22T21:54:01.410 INFO:teuthology.orchestra.run.trial093.stdout:Unpacking ceph (19.2.3-689-gf90f4eae-1jammy) ... 2026-02-22T21:54:01.424 INFO:teuthology.orchestra.run.trial093.stdout:Selecting previously unselected package ceph-fuse. 2026-02-22T21:54:01.432 INFO:teuthology.orchestra.run.trial093.stdout:Preparing to unpack .../062-ceph-fuse_19.2.3-689-gf90f4eae-1jammy_amd64.deb ... 2026-02-22T21:54:01.432 INFO:teuthology.orchestra.run.trial093.stdout:Unpacking ceph-fuse (19.2.3-689-gf90f4eae-1jammy) ... 2026-02-22T21:54:01.455 INFO:teuthology.orchestra.run.trial093.stdout:Selecting previously unselected package ceph-mds. 2026-02-22T21:54:01.464 INFO:teuthology.orchestra.run.trial093.stdout:Preparing to unpack .../063-ceph-mds_19.2.3-689-gf90f4eae-1jammy_amd64.deb ... 2026-02-22T21:54:01.464 INFO:teuthology.orchestra.run.trial093.stdout:Unpacking ceph-mds (19.2.3-689-gf90f4eae-1jammy) ... 2026-02-22T21:54:01.498 INFO:teuthology.orchestra.run.trial093.stdout:Selecting previously unselected package cephadm. 2026-02-22T21:54:01.505 INFO:teuthology.orchestra.run.trial093.stdout:Preparing to unpack .../064-cephadm_19.2.3-689-gf90f4eae-1jammy_amd64.deb ... 2026-02-22T21:54:01.506 INFO:teuthology.orchestra.run.trial093.stdout:Unpacking cephadm (19.2.3-689-gf90f4eae-1jammy) ... 2026-02-22T21:54:01.521 INFO:teuthology.orchestra.run.trial093.stdout:Selecting previously unselected package python3-asyncssh. 2026-02-22T21:54:01.529 INFO:teuthology.orchestra.run.trial093.stdout:Preparing to unpack .../065-python3-asyncssh_2.5.0-1ubuntu0.1_all.deb ... 2026-02-22T21:54:01.530 INFO:teuthology.orchestra.run.trial093.stdout:Unpacking python3-asyncssh (2.5.0-1ubuntu0.1) ... 2026-02-22T21:54:01.551 INFO:teuthology.orchestra.run.trial093.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2026-02-22T21:54:01.559 INFO:teuthology.orchestra.run.trial093.stdout:Preparing to unpack .../066-ceph-mgr-cephadm_19.2.3-689-gf90f4eae-1jammy_all.deb ... 2026-02-22T21:54:01.560 INFO:teuthology.orchestra.run.trial093.stdout:Unpacking ceph-mgr-cephadm (19.2.3-689-gf90f4eae-1jammy) ... 2026-02-22T21:54:01.580 INFO:teuthology.orchestra.run.trial093.stdout:Selecting previously unselected package python3-repoze.lru. 2026-02-22T21:54:01.588 INFO:teuthology.orchestra.run.trial093.stdout:Preparing to unpack .../067-python3-repoze.lru_0.7-2_all.deb ... 2026-02-22T21:54:01.588 INFO:teuthology.orchestra.run.trial093.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2026-02-22T21:54:01.603 INFO:teuthology.orchestra.run.trial093.stdout:Selecting previously unselected package python3-routes. 2026-02-22T21:54:01.611 INFO:teuthology.orchestra.run.trial093.stdout:Preparing to unpack .../068-python3-routes_2.5.1-1ubuntu1_all.deb ... 2026-02-22T21:54:01.611 INFO:teuthology.orchestra.run.trial093.stdout:Unpacking python3-routes (2.5.1-1ubuntu1) ... 2026-02-22T21:54:01.631 INFO:teuthology.orchestra.run.trial093.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2026-02-22T21:54:01.639 INFO:teuthology.orchestra.run.trial093.stdout:Preparing to unpack .../069-ceph-mgr-dashboard_19.2.3-689-gf90f4eae-1jammy_all.deb ... 2026-02-22T21:54:01.640 INFO:teuthology.orchestra.run.trial093.stdout:Unpacking ceph-mgr-dashboard (19.2.3-689-gf90f4eae-1jammy) ... 2026-02-22T21:54:01.846 INFO:teuthology.orchestra.run.trial093.stdout:Selecting previously unselected package python3-sklearn-lib:amd64. 2026-02-22T21:54:01.854 INFO:teuthology.orchestra.run.trial093.stdout:Preparing to unpack .../070-python3-sklearn-lib_0.23.2-5ubuntu6_amd64.deb ... 2026-02-22T21:54:01.855 INFO:teuthology.orchestra.run.trial093.stdout:Unpacking python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2026-02-22T21:54:01.893 INFO:teuthology.orchestra.run.trial093.stdout:Selecting previously unselected package python3-joblib. 2026-02-22T21:54:01.901 INFO:teuthology.orchestra.run.trial093.stdout:Preparing to unpack .../071-python3-joblib_0.17.0-4ubuntu1_all.deb ... 2026-02-22T21:54:01.901 INFO:teuthology.orchestra.run.trial093.stdout:Unpacking python3-joblib (0.17.0-4ubuntu1) ... 2026-02-22T21:54:01.929 INFO:teuthology.orchestra.run.trial093.stdout:Selecting previously unselected package python3-threadpoolctl. 2026-02-22T21:54:01.937 INFO:teuthology.orchestra.run.trial093.stdout:Preparing to unpack .../072-python3-threadpoolctl_3.1.0-1_all.deb ... 2026-02-22T21:54:01.938 INFO:teuthology.orchestra.run.trial093.stdout:Unpacking python3-threadpoolctl (3.1.0-1) ... 2026-02-22T21:54:01.953 INFO:teuthology.orchestra.run.trial093.stdout:Selecting previously unselected package python3-sklearn. 2026-02-22T21:54:01.961 INFO:teuthology.orchestra.run.trial093.stdout:Preparing to unpack .../073-python3-sklearn_0.23.2-5ubuntu6_all.deb ... 2026-02-22T21:54:01.961 INFO:teuthology.orchestra.run.trial093.stdout:Unpacking python3-sklearn (0.23.2-5ubuntu6) ... 2026-02-22T21:54:02.045 INFO:teuthology.orchestra.run.trial093.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2026-02-22T21:54:02.053 INFO:teuthology.orchestra.run.trial093.stdout:Preparing to unpack .../074-ceph-mgr-diskprediction-local_19.2.3-689-gf90f4eae-1jammy_all.deb ... 2026-02-22T21:54:02.053 INFO:teuthology.orchestra.run.trial093.stdout:Unpacking ceph-mgr-diskprediction-local (19.2.3-689-gf90f4eae-1jammy) ... 2026-02-22T21:54:02.224 INFO:teuthology.orchestra.run.trial093.stdout:Selecting previously unselected package python3-cachetools. 2026-02-22T21:54:02.233 INFO:teuthology.orchestra.run.trial093.stdout:Preparing to unpack .../075-python3-cachetools_5.0.0-1_all.deb ... 2026-02-22T21:54:02.233 INFO:teuthology.orchestra.run.trial093.stdout:Unpacking python3-cachetools (5.0.0-1) ... 2026-02-22T21:54:02.249 INFO:teuthology.orchestra.run.trial093.stdout:Selecting previously unselected package python3-rsa. 2026-02-22T21:54:02.257 INFO:teuthology.orchestra.run.trial093.stdout:Preparing to unpack .../076-python3-rsa_4.8-1_all.deb ... 2026-02-22T21:54:02.258 INFO:teuthology.orchestra.run.trial093.stdout:Unpacking python3-rsa (4.8-1) ... 2026-02-22T21:54:02.275 INFO:teuthology.orchestra.run.trial093.stdout:Selecting previously unselected package python3-google-auth. 2026-02-22T21:54:02.283 INFO:teuthology.orchestra.run.trial093.stdout:Preparing to unpack .../077-python3-google-auth_1.5.1-3_all.deb ... 2026-02-22T21:54:02.284 INFO:teuthology.orchestra.run.trial093.stdout:Unpacking python3-google-auth (1.5.1-3) ... 2026-02-22T21:54:02.301 INFO:teuthology.orchestra.run.trial093.stdout:Selecting previously unselected package python3-requests-oauthlib. 2026-02-22T21:54:02.310 INFO:teuthology.orchestra.run.trial093.stdout:Preparing to unpack .../078-python3-requests-oauthlib_1.3.0+ds-0.1_all.deb ... 2026-02-22T21:54:02.310 INFO:teuthology.orchestra.run.trial093.stdout:Unpacking python3-requests-oauthlib (1.3.0+ds-0.1) ... 2026-02-22T21:54:02.326 INFO:teuthology.orchestra.run.trial093.stdout:Selecting previously unselected package python3-websocket. 2026-02-22T21:54:02.334 INFO:teuthology.orchestra.run.trial093.stdout:Preparing to unpack .../079-python3-websocket_1.2.3-1_all.deb ... 2026-02-22T21:54:02.335 INFO:teuthology.orchestra.run.trial093.stdout:Unpacking python3-websocket (1.2.3-1) ... 2026-02-22T21:54:02.352 INFO:teuthology.orchestra.run.trial093.stdout:Selecting previously unselected package python3-kubernetes. 2026-02-22T21:54:02.360 INFO:teuthology.orchestra.run.trial093.stdout:Preparing to unpack .../080-python3-kubernetes_12.0.1-1ubuntu1_all.deb ... 2026-02-22T21:54:02.361 INFO:teuthology.orchestra.run.trial093.stdout:Unpacking python3-kubernetes (12.0.1-1ubuntu1) ... 2026-02-22T21:54:02.457 INFO:teuthology.orchestra.run.trial093.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2026-02-22T21:54:02.466 INFO:teuthology.orchestra.run.trial093.stdout:Preparing to unpack .../081-ceph-mgr-k8sevents_19.2.3-689-gf90f4eae-1jammy_all.deb ... 2026-02-22T21:54:02.466 INFO:teuthology.orchestra.run.trial093.stdout:Unpacking ceph-mgr-k8sevents (19.2.3-689-gf90f4eae-1jammy) ... 2026-02-22T21:54:02.481 INFO:teuthology.orchestra.run.trial093.stdout:Selecting previously unselected package libonig5:amd64. 2026-02-22T21:54:02.489 INFO:teuthology.orchestra.run.trial093.stdout:Preparing to unpack .../082-libonig5_6.9.7.1-2build1_amd64.deb ... 2026-02-22T21:54:02.496 INFO:teuthology.orchestra.run.trial093.stdout:Unpacking libonig5:amd64 (6.9.7.1-2build1) ... 2026-02-22T21:54:02.512 INFO:teuthology.orchestra.run.trial093.stdout:Selecting previously unselected package libjq1:amd64. 2026-02-22T21:54:02.520 INFO:teuthology.orchestra.run.trial093.stdout:Preparing to unpack .../083-libjq1_1.6-2.1ubuntu3.1_amd64.deb ... 2026-02-22T21:54:02.521 INFO:teuthology.orchestra.run.trial093.stdout:Unpacking libjq1:amd64 (1.6-2.1ubuntu3.1) ... 2026-02-22T21:54:02.536 INFO:teuthology.orchestra.run.trial093.stdout:Selecting previously unselected package jq. 2026-02-22T21:54:02.545 INFO:teuthology.orchestra.run.trial093.stdout:Preparing to unpack .../084-jq_1.6-2.1ubuntu3.1_amd64.deb ... 2026-02-22T21:54:02.545 INFO:teuthology.orchestra.run.trial093.stdout:Unpacking jq (1.6-2.1ubuntu3.1) ... 2026-02-22T21:54:02.559 INFO:teuthology.orchestra.run.trial093.stdout:Selecting previously unselected package socat. 2026-02-22T21:54:02.568 INFO:teuthology.orchestra.run.trial093.stdout:Preparing to unpack .../085-socat_1.7.4.1-3ubuntu4_amd64.deb ... 2026-02-22T21:54:02.568 INFO:teuthology.orchestra.run.trial093.stdout:Unpacking socat (1.7.4.1-3ubuntu4) ... 2026-02-22T21:54:02.588 INFO:teuthology.orchestra.run.trial093.stdout:Selecting previously unselected package xmlstarlet. 2026-02-22T21:54:02.596 INFO:teuthology.orchestra.run.trial093.stdout:Preparing to unpack .../086-xmlstarlet_1.6.1-2.1_amd64.deb ... 2026-02-22T21:54:02.597 INFO:teuthology.orchestra.run.trial093.stdout:Unpacking xmlstarlet (1.6.1-2.1) ... 2026-02-22T21:54:02.633 INFO:teuthology.orchestra.run.trial093.stdout:Selecting previously unselected package ceph-test. 2026-02-22T21:54:02.641 INFO:teuthology.orchestra.run.trial093.stdout:Preparing to unpack .../087-ceph-test_19.2.3-689-gf90f4eae-1jammy_amd64.deb ... 2026-02-22T21:54:02.642 INFO:teuthology.orchestra.run.trial093.stdout:Unpacking ceph-test (19.2.3-689-gf90f4eae-1jammy) ... 2026-02-22T21:54:03.117 INFO:teuthology.orchestra.run.trial093.stdout:Selecting previously unselected package ceph-volume. 2026-02-22T21:54:03.125 INFO:teuthology.orchestra.run.trial093.stdout:Preparing to unpack .../088-ceph-volume_19.2.3-689-gf90f4eae-1jammy_all.deb ... 2026-02-22T21:54:03.126 INFO:teuthology.orchestra.run.trial093.stdout:Unpacking ceph-volume (19.2.3-689-gf90f4eae-1jammy) ... 2026-02-22T21:54:03.149 INFO:teuthology.orchestra.run.trial093.stdout:Selecting previously unselected package libcephfs-dev. 2026-02-22T21:54:03.158 INFO:teuthology.orchestra.run.trial093.stdout:Preparing to unpack .../089-libcephfs-dev_19.2.3-689-gf90f4eae-1jammy_amd64.deb ... 2026-02-22T21:54:03.158 INFO:teuthology.orchestra.run.trial093.stdout:Unpacking libcephfs-dev (19.2.3-689-gf90f4eae-1jammy) ... 2026-02-22T21:54:03.174 INFO:teuthology.orchestra.run.trial093.stdout:Selecting previously unselected package lua-socket:amd64. 2026-02-22T21:54:03.182 INFO:teuthology.orchestra.run.trial093.stdout:Preparing to unpack .../090-lua-socket_3.0~rc1+git+ac3201d-6_amd64.deb ... 2026-02-22T21:54:03.183 INFO:teuthology.orchestra.run.trial093.stdout:Unpacking lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2026-02-22T21:54:03.203 INFO:teuthology.orchestra.run.trial093.stdout:Selecting previously unselected package lua-sec:amd64. 2026-02-22T21:54:03.212 INFO:teuthology.orchestra.run.trial093.stdout:Preparing to unpack .../091-lua-sec_1.0.2-1_amd64.deb ... 2026-02-22T21:54:03.212 INFO:teuthology.orchestra.run.trial093.stdout:Unpacking lua-sec:amd64 (1.0.2-1) ... 2026-02-22T21:54:03.229 INFO:teuthology.orchestra.run.trial093.stdout:Selecting previously unselected package nvme-cli. 2026-02-22T21:54:03.238 INFO:teuthology.orchestra.run.trial093.stdout:Preparing to unpack .../092-nvme-cli_1.16-3ubuntu0.3_amd64.deb ... 2026-02-22T21:54:03.238 INFO:teuthology.orchestra.run.trial093.stdout:Unpacking nvme-cli (1.16-3ubuntu0.3) ... 2026-02-22T21:54:03.269 INFO:teuthology.orchestra.run.trial093.stdout:Selecting previously unselected package pkg-config. 2026-02-22T21:54:03.278 INFO:teuthology.orchestra.run.trial093.stdout:Preparing to unpack .../093-pkg-config_0.29.2-1ubuntu3_amd64.deb ... 2026-02-22T21:54:03.278 INFO:teuthology.orchestra.run.trial093.stdout:Unpacking pkg-config (0.29.2-1ubuntu3) ... 2026-02-22T21:54:03.294 INFO:teuthology.orchestra.run.trial093.stdout:Selecting previously unselected package python-asyncssh-doc. 2026-02-22T21:54:03.303 INFO:teuthology.orchestra.run.trial093.stdout:Preparing to unpack .../094-python-asyncssh-doc_2.5.0-1ubuntu0.1_all.deb ... 2026-02-22T21:54:03.303 INFO:teuthology.orchestra.run.trial093.stdout:Unpacking python-asyncssh-doc (2.5.0-1ubuntu0.1) ... 2026-02-22T21:54:03.333 INFO:teuthology.orchestra.run.trial093.stdout:Selecting previously unselected package python3-iniconfig. 2026-02-22T21:54:03.341 INFO:teuthology.orchestra.run.trial093.stdout:Preparing to unpack .../095-python3-iniconfig_1.1.1-2_all.deb ... 2026-02-22T21:54:03.342 INFO:teuthology.orchestra.run.trial093.stdout:Unpacking python3-iniconfig (1.1.1-2) ... 2026-02-22T21:54:03.356 INFO:teuthology.orchestra.run.trial093.stdout:Selecting previously unselected package python3-pastescript. 2026-02-22T21:54:03.365 INFO:teuthology.orchestra.run.trial093.stdout:Preparing to unpack .../096-python3-pastescript_2.0.2-4_all.deb ... 2026-02-22T21:54:03.365 INFO:teuthology.orchestra.run.trial093.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2026-02-22T21:54:03.383 INFO:teuthology.orchestra.run.trial093.stdout:Selecting previously unselected package python3-pluggy. 2026-02-22T21:54:03.391 INFO:teuthology.orchestra.run.trial093.stdout:Preparing to unpack .../097-python3-pluggy_0.13.0-7.1_all.deb ... 2026-02-22T21:54:03.392 INFO:teuthology.orchestra.run.trial093.stdout:Unpacking python3-pluggy (0.13.0-7.1) ... 2026-02-22T21:54:03.408 INFO:teuthology.orchestra.run.trial093.stdout:Selecting previously unselected package python3-psutil. 2026-02-22T21:54:03.416 INFO:teuthology.orchestra.run.trial093.stdout:Preparing to unpack .../098-python3-psutil_5.9.0-1build1_amd64.deb ... 2026-02-22T21:54:03.416 INFO:teuthology.orchestra.run.trial093.stdout:Unpacking python3-psutil (5.9.0-1build1) ... 2026-02-22T21:54:03.434 INFO:teuthology.orchestra.run.trial093.stdout:Selecting previously unselected package python3-py. 2026-02-22T21:54:03.443 INFO:teuthology.orchestra.run.trial093.stdout:Preparing to unpack .../099-python3-py_1.10.0-1_all.deb ... 2026-02-22T21:54:03.443 INFO:teuthology.orchestra.run.trial093.stdout:Unpacking python3-py (1.10.0-1) ... 2026-02-22T21:54:03.463 INFO:teuthology.orchestra.run.trial093.stdout:Selecting previously unselected package python3-pygments. 2026-02-22T21:54:03.471 INFO:teuthology.orchestra.run.trial093.stdout:Preparing to unpack .../100-python3-pygments_2.11.2+dfsg-2ubuntu0.1_all.deb ... 2026-02-22T21:54:03.472 INFO:teuthology.orchestra.run.trial093.stdout:Unpacking python3-pygments (2.11.2+dfsg-2ubuntu0.1) ... 2026-02-22T21:54:03.514 INFO:teuthology.orchestra.run.trial093.stdout:Selecting previously unselected package python3-pyinotify. 2026-02-22T21:54:03.522 INFO:teuthology.orchestra.run.trial093.stdout:Preparing to unpack .../101-python3-pyinotify_0.9.6-1.3_all.deb ... 2026-02-22T21:54:03.523 INFO:teuthology.orchestra.run.trial093.stdout:Unpacking python3-pyinotify (0.9.6-1.3) ... 2026-02-22T21:54:03.537 INFO:teuthology.orchestra.run.trial093.stdout:Selecting previously unselected package python3-toml. 2026-02-22T21:54:03.545 INFO:teuthology.orchestra.run.trial093.stdout:Preparing to unpack .../102-python3-toml_0.10.2-1_all.deb ... 2026-02-22T21:54:03.546 INFO:teuthology.orchestra.run.trial093.stdout:Unpacking python3-toml (0.10.2-1) ... 2026-02-22T21:54:03.561 INFO:teuthology.orchestra.run.trial093.stdout:Selecting previously unselected package python3-pytest. 2026-02-22T21:54:03.569 INFO:teuthology.orchestra.run.trial093.stdout:Preparing to unpack .../103-python3-pytest_6.2.5-1ubuntu2_all.deb ... 2026-02-22T21:54:03.570 INFO:teuthology.orchestra.run.trial093.stdout:Unpacking python3-pytest (6.2.5-1ubuntu2) ... 2026-02-22T21:54:03.592 INFO:teuthology.orchestra.run.trial093.stdout:Selecting previously unselected package python3-simplejson. 2026-02-22T21:54:03.600 INFO:teuthology.orchestra.run.trial093.stdout:Preparing to unpack .../104-python3-simplejson_3.17.6-1build1_amd64.deb ... 2026-02-22T21:54:03.601 INFO:teuthology.orchestra.run.trial093.stdout:Unpacking python3-simplejson (3.17.6-1build1) ... 2026-02-22T21:54:03.618 INFO:teuthology.orchestra.run.trial093.stdout:Selecting previously unselected package qttranslations5-l10n. 2026-02-22T21:54:03.627 INFO:teuthology.orchestra.run.trial093.stdout:Preparing to unpack .../105-qttranslations5-l10n_5.15.3-1_all.deb ... 2026-02-22T21:54:03.627 INFO:teuthology.orchestra.run.trial093.stdout:Unpacking qttranslations5-l10n (5.15.3-1) ... 2026-02-22T21:54:03.695 INFO:teuthology.orchestra.run.trial093.stdout:Selecting previously unselected package radosgw. 2026-02-22T21:54:03.704 INFO:teuthology.orchestra.run.trial093.stdout:Preparing to unpack .../106-radosgw_19.2.3-689-gf90f4eae-1jammy_amd64.deb ... 2026-02-22T21:54:03.704 INFO:teuthology.orchestra.run.trial093.stdout:Unpacking radosgw (19.2.3-689-gf90f4eae-1jammy) ... 2026-02-22T21:54:03.819 INFO:teuthology.orchestra.run.trial093.stdout:Selecting previously unselected package rbd-fuse. 2026-02-22T21:54:03.827 INFO:teuthology.orchestra.run.trial093.stdout:Preparing to unpack .../107-rbd-fuse_19.2.3-689-gf90f4eae-1jammy_amd64.deb ... 2026-02-22T21:54:03.828 INFO:teuthology.orchestra.run.trial093.stdout:Unpacking rbd-fuse (19.2.3-689-gf90f4eae-1jammy) ... 2026-02-22T21:54:03.856 INFO:teuthology.orchestra.run.trial093.stdout:Setting up python3-iniconfig (1.1.1-2) ... 2026-02-22T21:54:03.924 INFO:teuthology.orchestra.run.trial093.stdout:Setting up libdouble-conversion3:amd64 (3.1.7-4) ... 2026-02-22T21:54:03.925 INFO:teuthology.orchestra.run.trial093.stdout:Setting up nvme-cli (1.16-3ubuntu0.3) ... 2026-02-22T21:54:03.995 INFO:teuthology.orchestra.run.trial093.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service → /lib/systemd/system/nvmefc-boot-connections.service. 2026-02-22T21:54:04.250 INFO:teuthology.orchestra.run.trial093.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmf-autoconnect.service → /lib/systemd/system/nvmf-autoconnect.service. 2026-02-22T21:54:04.597 INFO:teuthology.orchestra.run.trial093.stdout:nvmf-connect.target is a disabled or a static unit, not starting it. 2026-02-22T21:54:04.608 INFO:teuthology.orchestra.run.trial093.stdout:Setting up cephadm (19.2.3-689-gf90f4eae-1jammy) ... 2026-02-22T21:54:04.652 INFO:teuthology.orchestra.run.trial093.stdout:Adding system user cephadm....done 2026-02-22T21:54:04.662 INFO:teuthology.orchestra.run.trial093.stdout:Setting up python3-waitress (1.4.4-1.1ubuntu1.1) ... 2026-02-22T21:54:04.735 INFO:teuthology.orchestra.run.trial093.stdout:Setting up python3-jaraco.classes (3.2.1-3) ... 2026-02-22T21:54:04.805 INFO:teuthology.orchestra.run.trial093.stdout:Setting up python-asyncssh-doc (2.5.0-1ubuntu0.1) ... 2026-02-22T21:54:04.806 INFO:teuthology.orchestra.run.trial093.stdout:Setting up python3-jaraco.functools (3.4.0-2) ... 2026-02-22T21:54:04.878 INFO:teuthology.orchestra.run.trial093.stdout:Setting up python3-repoze.lru (0.7-2) ... 2026-02-22T21:54:04.953 INFO:teuthology.orchestra.run.trial093.stdout:Setting up liboath0:amd64 (2.6.7-3ubuntu0.1) ... 2026-02-22T21:54:04.954 INFO:teuthology.orchestra.run.trial093.stdout:Setting up python3-py (1.10.0-1) ... 2026-02-22T21:54:05.049 INFO:teuthology.orchestra.run.trial093.stdout:Setting up python3-joblib (0.17.0-4ubuntu1) ... 2026-02-22T21:54:05.171 INFO:teuthology.orchestra.run.trial093.stdout:Setting up python3-cachetools (5.0.0-1) ... 2026-02-22T21:54:05.247 INFO:teuthology.orchestra.run.trial093.stdout:Setting up unzip (6.0-26ubuntu3.2) ... 2026-02-22T21:54:05.258 INFO:teuthology.orchestra.run.trial093.stdout:Setting up python3-pyinotify (0.9.6-1.3) ... 2026-02-22T21:54:05.337 INFO:teuthology.orchestra.run.trial093.stdout:Setting up python3-threadpoolctl (3.1.0-1) ... 2026-02-22T21:54:05.411 INFO:teuthology.orchestra.run.trial093.stdout:Setting up python3-ceph-argparse (19.2.3-689-gf90f4eae-1jammy) ... 2026-02-22T21:54:05.478 INFO:teuthology.orchestra.run.trial093.stdout:Setting up python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2026-02-22T21:54:05.479 INFO:teuthology.orchestra.run.trial093.stdout:Setting up libnbd0 (1.10.5-1) ... 2026-02-22T21:54:05.480 INFO:teuthology.orchestra.run.trial093.stdout:Setting up lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2026-02-22T21:54:05.481 INFO:teuthology.orchestra.run.trial093.stdout:Setting up libreadline-dev:amd64 (8.1.2-1) ... 2026-02-22T21:54:05.482 INFO:teuthology.orchestra.run.trial093.stdout:Setting up libfuse2:amd64 (2.9.9-5ubuntu3) ... 2026-02-22T21:54:05.483 INFO:teuthology.orchestra.run.trial093.stdout:Setting up lua5.1 (5.1.5-8.1build4) ... 2026-02-22T21:54:05.499 INFO:teuthology.orchestra.run.trial093.stdout:update-alternatives: using /usr/bin/lua5.1 to provide /usr/bin/lua (lua-interpreter) in auto mode 2026-02-22T21:54:05.501 INFO:teuthology.orchestra.run.trial093.stdout:update-alternatives: using /usr/bin/luac5.1 to provide /usr/bin/luac (lua-compiler) in auto mode 2026-02-22T21:54:05.502 INFO:teuthology.orchestra.run.trial093.stdout:Setting up libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2026-02-22T21:54:05.503 INFO:teuthology.orchestra.run.trial093.stdout:Setting up python3-psutil (5.9.0-1build1) ... 2026-02-22T21:54:05.625 INFO:teuthology.orchestra.run.trial093.stdout:Setting up python3-natsort (8.0.2-1) ... 2026-02-22T21:54:05.701 INFO:teuthology.orchestra.run.trial093.stdout:Setting up python3-routes (2.5.1-1ubuntu1) ... 2026-02-22T21:54:05.778 INFO:teuthology.orchestra.run.trial093.stdout:Setting up python3-simplejson (3.17.6-1build1) ... 2026-02-22T21:54:05.864 INFO:teuthology.orchestra.run.trial093.stdout:Setting up zip (3.0-12build2) ... 2026-02-22T21:54:05.865 INFO:teuthology.orchestra.run.trial093.stdout:Setting up python3-pygments (2.11.2+dfsg-2ubuntu0.1) ... 2026-02-22T21:54:06.116 INFO:teuthology.orchestra.run.trial093.stdout:Setting up python3-tempita (0.5.2-6ubuntu1) ... 2026-02-22T21:54:06.192 INFO:teuthology.orchestra.run.trial093.stdout:Setting up python-pastedeploy-tpl (2.1.1-1) ... 2026-02-22T21:54:06.193 INFO:teuthology.orchestra.run.trial093.stdout:Setting up qttranslations5-l10n (5.15.3-1) ... 2026-02-22T21:54:06.194 INFO:teuthology.orchestra.run.trial093.stdout:Setting up python3-wcwidth (0.2.5+dfsg1-1) ... 2026-02-22T21:54:06.286 INFO:teuthology.orchestra.run.trial093.stdout:Setting up python3-asyncssh (2.5.0-1ubuntu0.1) ... 2026-02-22T21:54:06.410 INFO:teuthology.orchestra.run.trial093.stdout:Setting up python3-paste (3.5.0+dfsg1-1) ... 2026-02-22T21:54:06.540 INFO:teuthology.orchestra.run.trial093.stdout:Setting up python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2026-02-22T21:54:06.621 INFO:teuthology.orchestra.run.trial093.stdout:Setting up python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.3) ... 2026-02-22T21:54:06.725 INFO:teuthology.orchestra.run.trial093.stdout:Setting up python3-jaraco.text (3.6.0-2) ... 2026-02-22T21:54:06.799 INFO:teuthology.orchestra.run.trial093.stdout:Setting up socat (1.7.4.1-3ubuntu4) ... 2026-02-22T21:54:06.800 INFO:teuthology.orchestra.run.trial093.stdout:Setting up python3-ceph-common (19.2.3-689-gf90f4eae-1jammy) ... 2026-02-22T21:54:06.883 INFO:teuthology.orchestra.run.trial093.stdout:Setting up python3-sklearn (0.23.2-5ubuntu6) ... 2026-02-22T21:54:07.375 INFO:teuthology.orchestra.run.trial093.stdout:Setting up pkg-config (0.29.2-1ubuntu3) ... 2026-02-22T21:54:07.397 INFO:teuthology.orchestra.run.trial093.stdout:Setting up libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2026-02-22T21:54:07.402 INFO:teuthology.orchestra.run.trial093.stdout:Setting up python3-toml (0.10.2-1) ... 2026-02-22T21:54:07.480 INFO:teuthology.orchestra.run.trial093.stdout:Setting up librdkafka1:amd64 (1.8.0-1build1) ... 2026-02-22T21:54:07.481 INFO:teuthology.orchestra.run.trial093.stdout:Setting up xmlstarlet (1.6.1-2.1) ... 2026-02-22T21:54:07.482 INFO:teuthology.orchestra.run.trial093.stdout:Setting up python3-pluggy (0.13.0-7.1) ... 2026-02-22T21:54:07.562 INFO:teuthology.orchestra.run.trial093.stdout:Setting up python3-zc.lockfile (2.0-1) ... 2026-02-22T21:54:07.632 INFO:teuthology.orchestra.run.trial093.stdout:Setting up libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2026-02-22T21:54:07.633 INFO:teuthology.orchestra.run.trial093.stdout:Setting up python3-rsa (4.8-1) ... 2026-02-22T21:54:07.709 INFO:teuthology.orchestra.run.trial093.stdout:Setting up python3-singledispatch (3.4.0.3-3) ... 2026-02-22T21:54:07.781 INFO:teuthology.orchestra.run.trial093.stdout:Setting up python3-logutils (0.3.3-8) ... 2026-02-22T21:54:07.856 INFO:teuthology.orchestra.run.trial093.stdout:Setting up python3-tempora (4.1.2-1) ... 2026-02-22T21:54:07.933 INFO:teuthology.orchestra.run.trial093.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2026-02-22T21:54:08.004 INFO:teuthology.orchestra.run.trial093.stdout:Setting up python3-prettytable (2.5.0-2) ... 2026-02-22T21:54:08.083 INFO:teuthology.orchestra.run.trial093.stdout:Setting up liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2026-02-22T21:54:08.083 INFO:teuthology.orchestra.run.trial093.stdout:Setting up python3-websocket (1.2.3-1) ... 2026-02-22T21:54:08.167 INFO:teuthology.orchestra.run.trial093.stdout:Setting up libonig5:amd64 (6.9.7.1-2build1) ... 2026-02-22T21:54:08.168 INFO:teuthology.orchestra.run.trial093.stdout:Setting up python3-requests-oauthlib (1.3.0+ds-0.1) ... 2026-02-22T21:54:08.246 INFO:teuthology.orchestra.run.trial093.stdout:Setting up python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2026-02-22T21:54:08.326 INFO:teuthology.orchestra.run.trial093.stdout:Setting up python3-webob (1:1.8.6-1.1ubuntu0.1) ... 2026-02-22T21:54:08.409 INFO:teuthology.orchestra.run.trial093.stdout:Setting up python3-jaraco.collections (3.4.0-2) ... 2026-02-22T21:54:08.484 INFO:teuthology.orchestra.run.trial093.stdout:Setting up liblua5.3-dev:amd64 (5.3.6-1build1) ... 2026-02-22T21:54:08.485 INFO:teuthology.orchestra.run.trial093.stdout:Setting up lua-sec:amd64 (1.0.2-1) ... 2026-02-22T21:54:08.486 INFO:teuthology.orchestra.run.trial093.stdout:Setting up libjq1:amd64 (1.6-2.1ubuntu3.1) ... 2026-02-22T21:54:08.486 INFO:teuthology.orchestra.run.trial093.stdout:Setting up python3-pytest (6.2.5-1ubuntu2) ... 2026-02-22T21:54:08.622 INFO:teuthology.orchestra.run.trial093.stdout:Setting up python3-pastedeploy (2.1.1-1) ... 2026-02-22T21:54:08.697 INFO:teuthology.orchestra.run.trial093.stdout:Setting up lua-any (27ubuntu1) ... 2026-02-22T21:54:08.697 INFO:teuthology.orchestra.run.trial093.stdout:Setting up python3-portend (3.0.0-1) ... 2026-02-22T21:54:08.771 INFO:teuthology.orchestra.run.trial093.stdout:Setting up libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2026-02-22T21:54:08.772 INFO:teuthology.orchestra.run.trial093.stdout:Setting up python3-google-auth (1.5.1-3) ... 2026-02-22T21:54:08.854 INFO:teuthology.orchestra.run.trial093.stdout:Setting up jq (1.6-2.1ubuntu3.1) ... 2026-02-22T21:54:08.855 INFO:teuthology.orchestra.run.trial093.stdout:Setting up python3-webtest (2.0.35-1) ... 2026-02-22T21:54:08.932 INFO:teuthology.orchestra.run.trial093.stdout:Setting up python3-cherrypy3 (18.6.1-4) ... 2026-02-22T21:54:09.066 INFO:teuthology.orchestra.run.trial093.stdout:Setting up python3-pastescript (2.0.2-4) ... 2026-02-22T21:54:09.153 INFO:teuthology.orchestra.run.trial093.stdout:Setting up python3-pecan (1.3.3-4ubuntu2) ... 2026-02-22T21:54:09.268 INFO:teuthology.orchestra.run.trial093.stdout:Setting up libthrift-0.16.0:amd64 (0.16.0-2) ... 2026-02-22T21:54:09.269 INFO:teuthology.orchestra.run.trial093.stdout:Setting up librados2 (19.2.3-689-gf90f4eae-1jammy) ... 2026-02-22T21:54:09.269 INFO:teuthology.orchestra.run.trial093.stdout:Setting up libsqlite3-mod-ceph (19.2.3-689-gf90f4eae-1jammy) ... 2026-02-22T21:54:09.270 INFO:teuthology.orchestra.run.trial093.stdout:Setting up python3-kubernetes (12.0.1-1ubuntu1) ... 2026-02-22T21:54:09.786 INFO:teuthology.orchestra.run.trial093.stdout:Setting up luarocks (3.8.0+dfsg1-1) ... 2026-02-22T21:54:09.797 INFO:teuthology.orchestra.run.trial093.stdout:Setting up libcephfs2 (19.2.3-689-gf90f4eae-1jammy) ... 2026-02-22T21:54:09.797 INFO:teuthology.orchestra.run.trial093.stdout:Setting up libradosstriper1 (19.2.3-689-gf90f4eae-1jammy) ... 2026-02-22T21:54:09.798 INFO:teuthology.orchestra.run.trial093.stdout:Setting up librbd1 (19.2.3-689-gf90f4eae-1jammy) ... 2026-02-22T21:54:09.799 INFO:teuthology.orchestra.run.trial093.stdout:Setting up ceph-mgr-modules-core (19.2.3-689-gf90f4eae-1jammy) ... 2026-02-22T21:54:09.800 INFO:teuthology.orchestra.run.trial093.stdout:Setting up ceph-fuse (19.2.3-689-gf90f4eae-1jammy) ... 2026-02-22T21:54:09.871 INFO:teuthology.orchestra.run.trial093.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target → /lib/systemd/system/ceph-fuse.target. 2026-02-22T21:54:09.871 INFO:teuthology.orchestra.run.trial093.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target → /lib/systemd/system/ceph-fuse.target. 2026-02-22T21:54:10.238 INFO:teuthology.orchestra.run.trial093.stdout:Setting up libcephfs-dev (19.2.3-689-gf90f4eae-1jammy) ... 2026-02-22T21:54:10.238 INFO:teuthology.orchestra.run.trial093.stdout:Setting up python3-rados (19.2.3-689-gf90f4eae-1jammy) ... 2026-02-22T21:54:10.239 INFO:teuthology.orchestra.run.trial093.stdout:Setting up librgw2 (19.2.3-689-gf90f4eae-1jammy) ... 2026-02-22T21:54:10.240 INFO:teuthology.orchestra.run.trial093.stdout:Setting up python3-rbd (19.2.3-689-gf90f4eae-1jammy) ... 2026-02-22T21:54:10.241 INFO:teuthology.orchestra.run.trial093.stdout:Setting up rbd-fuse (19.2.3-689-gf90f4eae-1jammy) ... 2026-02-22T21:54:10.242 INFO:teuthology.orchestra.run.trial093.stdout:Setting up python3-rgw (19.2.3-689-gf90f4eae-1jammy) ... 2026-02-22T21:54:10.243 INFO:teuthology.orchestra.run.trial093.stdout:Setting up python3-cephfs (19.2.3-689-gf90f4eae-1jammy) ... 2026-02-22T21:54:10.244 INFO:teuthology.orchestra.run.trial093.stdout:Setting up ceph-common (19.2.3-689-gf90f4eae-1jammy) ... 2026-02-22T21:54:10.270 INFO:teuthology.orchestra.run.trial093.stdout:Adding group ceph....done 2026-02-22T21:54:10.308 INFO:teuthology.orchestra.run.trial093.stdout:Adding system user ceph....done 2026-02-22T21:54:10.318 INFO:teuthology.orchestra.run.trial093.stdout:Setting system user ceph properties....done 2026-02-22T21:54:10.324 INFO:teuthology.orchestra.run.trial093.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2026-02-22T21:54:10.410 INFO:teuthology.orchestra.run.trial093.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /lib/systemd/system/ceph.target. 2026-02-22T21:54:10.683 INFO:teuthology.orchestra.run.trial093.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service → /lib/systemd/system/rbdmap.service. 2026-02-22T21:54:11.055 INFO:teuthology.orchestra.run.trial093.stdout:Setting up ceph-test (19.2.3-689-gf90f4eae-1jammy) ... 2026-02-22T21:54:11.056 INFO:teuthology.orchestra.run.trial093.stdout:Setting up radosgw (19.2.3-689-gf90f4eae-1jammy) ... 2026-02-22T21:54:11.303 INFO:teuthology.orchestra.run.trial093.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /lib/systemd/system/ceph-radosgw.target. 2026-02-22T21:54:11.303 INFO:teuthology.orchestra.run.trial093.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /lib/systemd/system/ceph-radosgw.target. 2026-02-22T21:54:11.649 INFO:teuthology.orchestra.run.trial093.stdout:Setting up ceph-base (19.2.3-689-gf90f4eae-1jammy) ... 2026-02-22T21:54:11.767 INFO:teuthology.orchestra.run.trial093.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /lib/systemd/system/ceph-crash.service. 2026-02-22T21:54:12.127 INFO:teuthology.orchestra.run.trial093.stdout:Setting up ceph-mds (19.2.3-689-gf90f4eae-1jammy) ... 2026-02-22T21:54:12.186 INFO:teuthology.orchestra.run.trial093.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /lib/systemd/system/ceph-mds.target. 2026-02-22T21:54:12.187 INFO:teuthology.orchestra.run.trial093.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /lib/systemd/system/ceph-mds.target. 2026-02-22T21:54:12.527 INFO:teuthology.orchestra.run.trial093.stdout:Setting up ceph-mgr (19.2.3-689-gf90f4eae-1jammy) ... 2026-02-22T21:54:12.607 INFO:teuthology.orchestra.run.trial093.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /lib/systemd/system/ceph-mgr.target. 2026-02-22T21:54:12.607 INFO:teuthology.orchestra.run.trial093.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /lib/systemd/system/ceph-mgr.target. 2026-02-22T21:54:12.950 INFO:teuthology.orchestra.run.trial093.stdout:Setting up ceph-osd (19.2.3-689-gf90f4eae-1jammy) ... 2026-02-22T21:54:13.032 INFO:teuthology.orchestra.run.trial093.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /lib/systemd/system/ceph-osd.target. 2026-02-22T21:54:13.032 INFO:teuthology.orchestra.run.trial093.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /lib/systemd/system/ceph-osd.target. 2026-02-22T21:54:13.401 INFO:teuthology.orchestra.run.trial093.stdout:Setting up ceph-mgr-k8sevents (19.2.3-689-gf90f4eae-1jammy) ... 2026-02-22T21:54:13.402 INFO:teuthology.orchestra.run.trial093.stdout:Setting up ceph-mgr-diskprediction-local (19.2.3-689-gf90f4eae-1jammy) ... 2026-02-22T21:54:13.414 INFO:teuthology.orchestra.run.trial093.stdout:Setting up ceph-mon (19.2.3-689-gf90f4eae-1jammy) ... 2026-02-22T21:54:13.491 INFO:teuthology.orchestra.run.trial093.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /lib/systemd/system/ceph-mon.target. 2026-02-22T21:54:13.491 INFO:teuthology.orchestra.run.trial093.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /lib/systemd/system/ceph-mon.target. 2026-02-22T21:54:13.852 INFO:teuthology.orchestra.run.trial093.stdout:Setting up ceph-mgr-cephadm (19.2.3-689-gf90f4eae-1jammy) ... 2026-02-22T21:54:13.863 INFO:teuthology.orchestra.run.trial093.stdout:Setting up ceph (19.2.3-689-gf90f4eae-1jammy) ... 2026-02-22T21:54:13.864 INFO:teuthology.orchestra.run.trial093.stdout:Setting up ceph-mgr-dashboard (19.2.3-689-gf90f4eae-1jammy) ... 2026-02-22T21:54:13.875 INFO:teuthology.orchestra.run.trial093.stdout:Setting up ceph-volume (19.2.3-689-gf90f4eae-1jammy) ... 2026-02-22T21:54:13.986 INFO:teuthology.orchestra.run.trial093.stdout:Processing triggers for mailcap (3.70+nmu1ubuntu1) ... 2026-02-22T21:54:13.994 INFO:teuthology.orchestra.run.trial093.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.13) ... 2026-02-22T21:54:14.238 INFO:teuthology.orchestra.run.trial093.stdout:Processing triggers for man-db (2.10.2-1) ... 2026-02-22T21:54:14.325 INFO:teuthology.orchestra.run.trial093.stdout:Processing triggers for install-info (6.8-4build1) ... 2026-02-22T21:54:14.831 INFO:teuthology.orchestra.run.trial093.stdout: 2026-02-22T21:54:14.831 INFO:teuthology.orchestra.run.trial093.stdout:Running kernel seems to be up-to-date. 2026-02-22T21:54:14.831 INFO:teuthology.orchestra.run.trial093.stdout: 2026-02-22T21:54:14.831 INFO:teuthology.orchestra.run.trial093.stdout:The processor microcode seems to be up-to-date. 2026-02-22T21:54:14.832 INFO:teuthology.orchestra.run.trial093.stdout: 2026-02-22T21:54:14.832 INFO:teuthology.orchestra.run.trial093.stdout:No services need to be restarted. 2026-02-22T21:54:14.832 INFO:teuthology.orchestra.run.trial093.stdout: 2026-02-22T21:54:14.832 INFO:teuthology.orchestra.run.trial093.stdout:No containers need to be restarted. 2026-02-22T21:54:14.832 INFO:teuthology.orchestra.run.trial093.stdout: 2026-02-22T21:54:14.832 INFO:teuthology.orchestra.run.trial093.stdout:No user sessions are running outdated binaries. 2026-02-22T21:54:14.832 INFO:teuthology.orchestra.run.trial093.stdout: 2026-02-22T21:54:14.832 INFO:teuthology.orchestra.run.trial093.stdout:No VM guests are running outdated hypervisor (qemu) binaries on this host. 2026-02-22T21:54:15.622 INFO:teuthology.orchestra.run.trial093.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2026-02-22T21:54:15.631 DEBUG:teuthology.parallel:result is None 2026-02-22T21:54:15.633 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=f90f4eaebb28b619cf76e80cbbe997656da69579 2026-02-22T21:54:15.797 DEBUG:teuthology.orchestra.run.trial093:> dpkg-query -W -f '${Version}' ceph 2026-02-22T21:54:15.808 INFO:teuthology.orchestra.run.trial093.stdout:19.2.3-689-gf90f4eae-1jammy 2026-02-22T21:54:15.809 INFO:teuthology.packaging:The installed version of ceph is 19.2.3-689-gf90f4eae-1jammy 2026-02-22T21:54:15.809 INFO:teuthology.task.install:The correct ceph version 19.2.3-689-gf90f4eae-1jammy is installed. 2026-02-22T21:54:15.811 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2026-02-22T21:54:15.811 DEBUG:teuthology.orchestra.run.trial093:> set -ex 2026-02-22T21:54:15.811 DEBUG:teuthology.orchestra.run.trial093:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2026-02-22T21:54:15.860 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2026-02-22T21:54:15.860 DEBUG:teuthology.orchestra.run.trial093:> set -ex 2026-02-22T21:54:15.860 DEBUG:teuthology.orchestra.run.trial093:> sudo dd of=/usr/bin/daemon-helper 2026-02-22T21:54:15.911 DEBUG:teuthology.orchestra.run.trial093:> sudo chmod a=rx -- /usr/bin/daemon-helper 2026-02-22T21:54:15.959 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2026-02-22T21:54:15.960 DEBUG:teuthology.orchestra.run.trial093:> set -ex 2026-02-22T21:54:15.960 DEBUG:teuthology.orchestra.run.trial093:> sudo dd of=/usr/bin/adjust-ulimits 2026-02-22T21:54:16.007 DEBUG:teuthology.orchestra.run.trial093:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2026-02-22T21:54:16.055 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2026-02-22T21:54:16.056 DEBUG:teuthology.orchestra.run.trial093:> set -ex 2026-02-22T21:54:16.056 DEBUG:teuthology.orchestra.run.trial093:> sudo dd of=/usr/bin/stdin-killer 2026-02-22T21:54:16.107 DEBUG:teuthology.orchestra.run.trial093:> sudo chmod a=rx -- /usr/bin/stdin-killer 2026-02-22T21:54:16.155 INFO:teuthology.run_tasks:Running task ceph... 2026-02-22T21:54:16.275 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2026-02-22T21:54:16.275 DEBUG:teuthology.orchestra.run.trial093:> sudo chmod 777 /var/log/ceph 2026-02-22T21:54:16.283 INFO:tasks.ceph:Disabling ceph logrotate... 2026-02-22T21:54:16.284 DEBUG:teuthology.orchestra.run.trial093:> sudo rm -f -- /etc/logrotate.d/ceph 2026-02-22T21:54:16.335 INFO:tasks.ceph:Creating extra log directories... 2026-02-22T21:54:16.335 DEBUG:teuthology.orchestra.run.trial093:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2026-02-22T21:54:16.389 INFO:tasks.ceph:Creating ceph cluster ceph... 2026-02-22T21:54:16.389 INFO:tasks.ceph:config {'conf': {'global': {'mon warn on pool no app': False}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'bluefs allocator': 'hybrid', 'bluestore allocator': 'hybrid', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2026-02-22T21:54:16.389 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/mnt/teuthology/yuriw-2026-02-22_19:35:38-rbd-wip-yuri6-testing-2026-02-22-1629-squid-distro-default-trial/64132', 'branch': 'wip-yuri6-testing-2026-02-22-1629-squid', 'description': 'rbd/singleton/{all/admin_socket conf/{disable-pool-app} objectstore/bluestore-hybrid openstack supported-random-distro$/{ubuntu_latest}}', 'email': 'yweinste@redhat.com', 'first_in_suite': False, 'flavor': 'default', 'job_id': '64132', 'kernel': {'branch': 'distro', 'kdb': True, 'sha1': 'distro'}, 'ktype': 'distro', 'last_in_suite': False, 'machine_type': 'trial', 'name': 'yuriw-2026-02-22_19:35:38-rbd-wip-yuri6-testing-2026-02-22-1629-squid-distro-default-trial', 'no_nested_subset': False, 'openstack': [{'volumes': {'count': 2, 'size': 30}}], 'os_type': 'ubuntu', 'os_version': '22.04', 'overrides': {'admin_socket': {'branch': 'wip-yuri6-testing-2026-02-22-1629-squid'}, 'ceph': {'conf': {'global': {'mon warn on pool no app': False}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'bluefs allocator': 'hybrid', 'bluestore allocator': 'hybrid', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)'], 'sha1': 'f90f4eaebb28b619cf76e80cbbe997656da69579'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': 'f90f4eaebb28b619cf76e80cbbe997656da69579'}}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-yuri6-testing-2026-02-22-1629-squid', 'sha1': 'f90f4eaebb28b619cf76e80cbbe997656da69579'}}, 'owner': 'scheduled_yuriw@soko04.front.sepia.ceph.com', 'priority': 69, 'repo': 'https://github.com/ceph/ceph-ci.git', 'roles': [['mon.a', 'mgr.x', 'osd.0', 'osd.1', 'client.0']], 'seed': 9196, 'sha1': 'f90f4eaebb28b619cf76e80cbbe997656da69579', 'sleep_before_teardown': 0, 'subset': '111/120000', 'suite': 'rbd', 'suite_branch': 'wip-yuri6-testing-2026-02-22-1629-squid', 'suite_path': '/home/teuthworker/src/github.com_ceph_ceph-c_f90f4eaebb28b619cf76e80cbbe997656da69579/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://github.com/ceph/ceph-ci.git', 'suite_sha1': 'f90f4eaebb28b619cf76e80cbbe997656da69579', 'targets': {'trial093.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBB9rVJr06cLSWEqqEYI8bgLPlRtQV1IKTC6sJrOEtwq1qELr+VK5JNLrAFT0oSVDspQUdFV/QbfNmN1hG5eDx04='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'branch': 'distro', 'kdb': True, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': None}, {'ceph': {'fs': 'xfs', 'conf': {'global': {'mon warn on pool no app': False}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'bluefs allocator': 'hybrid', 'bluestore allocator': 'hybrid', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'flavor': 'default', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)'], 'sha1': 'f90f4eaebb28b619cf76e80cbbe997656da69579', 'cluster': 'ceph'}}, {'workunit': {'clients': {'all': ['rbd/test_admin_socket.sh']}}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': []}, 'teuthology_branch': 'main', 'teuthology_sha1': 'fa17720d0088c3ac28e473468bfc79eeaff5cd38', 'timestamp': '2026-02-22_19:35:38', 'tube': 'trial', 'user': 'yuriw', 'verbose': True, 'worker_log': '/home/teuthworker/mnt/teuthology/worker_logs/dispatcher.trial.3318653'} 2026-02-22T21:54:16.390 DEBUG:teuthology.orchestra.run.trial093:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2026-02-22T21:54:16.427 DEBUG:teuthology.orchestra.run.trial093:> sudo install -d -m0777 -- /var/run/ceph 2026-02-22T21:54:16.476 DEBUG:teuthology.orchestra.run.trial093:> set -ex 2026-02-22T21:54:16.476 DEBUG:teuthology.orchestra.run.trial093:> dd if=/scratch_devs of=/dev/stdout 2026-02-22T21:54:16.525 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2026-02-22T21:54:16.525 DEBUG:teuthology.orchestra.run.trial093:> stat /dev/vg_nvme/lv_1 2026-02-22T21:54:16.571 INFO:teuthology.orchestra.run.trial093.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2026-02-22T21:54:16.572 INFO:teuthology.orchestra.run.trial093.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2026-02-22T21:54:16.572 INFO:teuthology.orchestra.run.trial093.stdout:Device: 5h/5d Inode: 902 Links: 1 2026-02-22T21:54:16.572 INFO:teuthology.orchestra.run.trial093.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2026-02-22T21:54:16.572 INFO:teuthology.orchestra.run.trial093.stdout:Access: 2026-02-22 21:53:32.227594872 +0000 2026-02-22T21:54:16.572 INFO:teuthology.orchestra.run.trial093.stdout:Modify: 2026-02-22 21:53:32.059597726 +0000 2026-02-22T21:54:16.572 INFO:teuthology.orchestra.run.trial093.stdout:Change: 2026-02-22 21:53:32.059597726 +0000 2026-02-22T21:54:16.572 INFO:teuthology.orchestra.run.trial093.stdout: Birth: - 2026-02-22T21:54:16.572 DEBUG:teuthology.orchestra.run.trial093:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2026-02-22T21:54:16.625 INFO:teuthology.orchestra.run.trial093.stderr:1+0 records in 2026-02-22T21:54:16.625 INFO:teuthology.orchestra.run.trial093.stderr:1+0 records out 2026-02-22T21:54:16.625 INFO:teuthology.orchestra.run.trial093.stderr:512 bytes copied, 0.000192578 s, 2.7 MB/s 2026-02-22T21:54:16.626 DEBUG:teuthology.orchestra.run.trial093:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2026-02-22T21:54:16.673 DEBUG:teuthology.orchestra.run.trial093:> stat /dev/vg_nvme/lv_2 2026-02-22T21:54:16.718 INFO:teuthology.orchestra.run.trial093.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2026-02-22T21:54:16.718 INFO:teuthology.orchestra.run.trial093.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2026-02-22T21:54:16.718 INFO:teuthology.orchestra.run.trial093.stdout:Device: 5h/5d Inode: 928 Links: 1 2026-02-22T21:54:16.718 INFO:teuthology.orchestra.run.trial093.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2026-02-22T21:54:16.718 INFO:teuthology.orchestra.run.trial093.stdout:Access: 2026-02-22 21:53:32.571589022 +0000 2026-02-22T21:54:16.718 INFO:teuthology.orchestra.run.trial093.stdout:Modify: 2026-02-22 21:53:32.399591945 +0000 2026-02-22T21:54:16.718 INFO:teuthology.orchestra.run.trial093.stdout:Change: 2026-02-22 21:53:32.399591945 +0000 2026-02-22T21:54:16.718 INFO:teuthology.orchestra.run.trial093.stdout: Birth: - 2026-02-22T21:54:16.719 DEBUG:teuthology.orchestra.run.trial093:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2026-02-22T21:54:16.766 INFO:teuthology.orchestra.run.trial093.stderr:1+0 records in 2026-02-22T21:54:16.766 INFO:teuthology.orchestra.run.trial093.stderr:1+0 records out 2026-02-22T21:54:16.766 INFO:teuthology.orchestra.run.trial093.stderr:512 bytes copied, 0.000178923 s, 2.9 MB/s 2026-02-22T21:54:16.767 DEBUG:teuthology.orchestra.run.trial093:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2026-02-22T21:54:16.816 DEBUG:teuthology.orchestra.run.trial093:> stat /dev/vg_nvme/lv_3 2026-02-22T21:54:16.862 INFO:teuthology.orchestra.run.trial093.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2026-02-22T21:54:16.863 INFO:teuthology.orchestra.run.trial093.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2026-02-22T21:54:16.863 INFO:teuthology.orchestra.run.trial093.stdout:Device: 5h/5d Inode: 952 Links: 1 2026-02-22T21:54:16.863 INFO:teuthology.orchestra.run.trial093.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2026-02-22T21:54:16.863 INFO:teuthology.orchestra.run.trial093.stdout:Access: 2026-02-22 21:53:32.919583107 +0000 2026-02-22T21:54:16.863 INFO:teuthology.orchestra.run.trial093.stdout:Modify: 2026-02-22 21:53:32.735586233 +0000 2026-02-22T21:54:16.863 INFO:teuthology.orchestra.run.trial093.stdout:Change: 2026-02-22 21:53:32.735586233 +0000 2026-02-22T21:54:16.863 INFO:teuthology.orchestra.run.trial093.stdout: Birth: - 2026-02-22T21:54:16.863 DEBUG:teuthology.orchestra.run.trial093:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2026-02-22T21:54:16.910 INFO:teuthology.orchestra.run.trial093.stderr:1+0 records in 2026-02-22T21:54:16.910 INFO:teuthology.orchestra.run.trial093.stderr:1+0 records out 2026-02-22T21:54:16.910 INFO:teuthology.orchestra.run.trial093.stderr:512 bytes copied, 0.000191907 s, 2.7 MB/s 2026-02-22T21:54:16.910 DEBUG:teuthology.orchestra.run.trial093:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2026-02-22T21:54:16.956 DEBUG:teuthology.orchestra.run.trial093:> stat /dev/vg_nvme/lv_4 2026-02-22T21:54:17.003 INFO:teuthology.orchestra.run.trial093.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2026-02-22T21:54:17.003 INFO:teuthology.orchestra.run.trial093.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2026-02-22T21:54:17.003 INFO:teuthology.orchestra.run.trial093.stdout:Device: 5h/5d Inode: 975 Links: 1 2026-02-22T21:54:17.003 INFO:teuthology.orchestra.run.trial093.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2026-02-22T21:54:17.003 INFO:teuthology.orchestra.run.trial093.stdout:Access: 2026-02-22 21:53:33.091580184 +0000 2026-02-22T21:54:17.003 INFO:teuthology.orchestra.run.trial093.stdout:Modify: 2026-02-22 21:53:33.087580249 +0000 2026-02-22T21:54:17.003 INFO:teuthology.orchestra.run.trial093.stdout:Change: 2026-02-22 21:53:33.087580249 +0000 2026-02-22T21:54:17.003 INFO:teuthology.orchestra.run.trial093.stdout: Birth: - 2026-02-22T21:54:17.004 DEBUG:teuthology.orchestra.run.trial093:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2026-02-22T21:54:17.050 INFO:teuthology.orchestra.run.trial093.stderr:1+0 records in 2026-02-22T21:54:17.051 INFO:teuthology.orchestra.run.trial093.stderr:1+0 records out 2026-02-22T21:54:17.051 INFO:teuthology.orchestra.run.trial093.stderr:512 bytes copied, 0.000176177 s, 2.9 MB/s 2026-02-22T21:54:17.052 DEBUG:teuthology.orchestra.run.trial093:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2026-02-22T21:54:17.101 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2'} 2026-02-22T21:54:17.101 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@trial093.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2'}} 2026-02-22T21:54:17.101 INFO:tasks.ceph:Generating config... 2026-02-22T21:54:17.104 INFO:tasks.ceph:[global] mon warn on pool no app = False 2026-02-22T21:54:17.104 INFO:tasks.ceph:[mgr] debug mgr = 20 2026-02-22T21:54:17.104 INFO:tasks.ceph:[mgr] debug ms = 1 2026-02-22T21:54:17.104 INFO:tasks.ceph:[mon] debug mon = 20 2026-02-22T21:54:17.104 INFO:tasks.ceph:[mon] debug ms = 1 2026-02-22T21:54:17.104 INFO:tasks.ceph:[mon] debug paxos = 20 2026-02-22T21:54:17.104 INFO:tasks.ceph:[osd] bluefs allocator = hybrid 2026-02-22T21:54:17.104 INFO:tasks.ceph:[osd] bluestore allocator = hybrid 2026-02-22T21:54:17.104 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2026-02-22T21:54:17.105 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2026-02-22T21:54:17.105 INFO:tasks.ceph:[osd] debug bluefs = 1/20 2026-02-22T21:54:17.105 INFO:tasks.ceph:[osd] debug bluestore = 1/20 2026-02-22T21:54:17.105 INFO:tasks.ceph:[osd] debug ms = 1 2026-02-22T21:54:17.105 INFO:tasks.ceph:[osd] debug osd = 20 2026-02-22T21:54:17.105 INFO:tasks.ceph:[osd] debug rocksdb = 4/10 2026-02-22T21:54:17.105 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2026-02-22T21:54:17.105 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2026-02-22T21:54:17.105 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2026-02-22T21:54:17.105 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2026-02-22T21:54:17.105 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2026-02-22T21:54:17.106 INFO:tasks.ceph:Setting up mon.a... 2026-02-22T21:54:17.106 DEBUG:teuthology.orchestra.run.trial093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2026-02-22T21:54:17.160 INFO:teuthology.orchestra.run.trial093.stdout:creating /etc/ceph/ceph.keyring 2026-02-22T21:54:17.163 DEBUG:teuthology.orchestra.run.trial093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2026-02-22T21:54:17.222 DEBUG:teuthology.orchestra.run.trial093:> sudo chmod 0644 /etc/ceph/ceph.keyring 2026-02-22T21:54:17.270 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '10.20.193.93')] 2026-02-22T21:54:17.270 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '10.20.193.93', 'mon warn on pool no app': False}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd mclock skip benchmark': 'true', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bluefs allocator': 'hybrid', 'bluestore allocator': 'hybrid', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false'}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok'}, 'mon.a': {}} 2026-02-22T21:54:17.271 DEBUG:teuthology.orchestra.run.trial093:> set -ex 2026-02-22T21:54:17.271 DEBUG:teuthology.orchestra.run.trial093:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2026-02-22T21:54:17.315 DEBUG:teuthology.orchestra.run.trial093:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 10.20.193.93 --print /home/ubuntu/cephtest/ceph.monmap 2026-02-22T21:54:17.376 INFO:teuthology.orchestra.run.trial093.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2026-02-22T21:54:17.376 INFO:teuthology.orchestra.run.trial093.stdout:monmaptool: generated fsid 63f41746-3baf-462b-9374-bd9179f37750 2026-02-22T21:54:17.376 INFO:teuthology.orchestra.run.trial093.stdout:setting min_mon_release = quincy 2026-02-22T21:54:17.376 INFO:teuthology.orchestra.run.trial093.stdout:epoch 0 2026-02-22T21:54:17.376 INFO:teuthology.orchestra.run.trial093.stdout:fsid 63f41746-3baf-462b-9374-bd9179f37750 2026-02-22T21:54:17.376 INFO:teuthology.orchestra.run.trial093.stdout:last_changed 2026-02-22T21:54:17.374621+0000 2026-02-22T21:54:17.376 INFO:teuthology.orchestra.run.trial093.stdout:created 2026-02-22T21:54:17.374621+0000 2026-02-22T21:54:17.376 INFO:teuthology.orchestra.run.trial093.stdout:min_mon_release 17 (quincy) 2026-02-22T21:54:17.376 INFO:teuthology.orchestra.run.trial093.stdout:election_strategy: 1 2026-02-22T21:54:17.377 INFO:teuthology.orchestra.run.trial093.stdout:0: [v2:10.20.193.93:3300/0,v1:10.20.193.93:6789/0] mon.a 2026-02-22T21:54:17.377 INFO:teuthology.orchestra.run.trial093.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (1 monitors) 2026-02-22T21:54:17.378 DEBUG:teuthology.orchestra.run.trial093:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2026-02-22T21:54:17.423 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 63f41746-3baf-462b-9374-bd9179f37750... 2026-02-22T21:54:17.424 DEBUG:teuthology.orchestra.run.trial093:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2026-02-22T21:54:17.483 INFO:teuthology.orchestra.run.trial093.stdout:[global] 2026-02-22T21:54:17.483 INFO:teuthology.orchestra.run.trial093.stdout: chdir = "" 2026-02-22T21:54:17.483 INFO:teuthology.orchestra.run.trial093.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2026-02-22T21:54:17.484 INFO:teuthology.orchestra.run.trial093.stdout: auth supported = cephx 2026-02-22T21:54:17.484 INFO:teuthology.orchestra.run.trial093.stdout: 2026-02-22T21:54:17.484 INFO:teuthology.orchestra.run.trial093.stdout: filestore xattr use omap = true 2026-02-22T21:54:17.484 INFO:teuthology.orchestra.run.trial093.stdout: 2026-02-22T21:54:17.484 INFO:teuthology.orchestra.run.trial093.stdout: mon clock drift allowed = 1.000 2026-02-22T21:54:17.484 INFO:teuthology.orchestra.run.trial093.stdout: 2026-02-22T21:54:17.484 INFO:teuthology.orchestra.run.trial093.stdout: osd crush chooseleaf type = 0 2026-02-22T21:54:17.484 INFO:teuthology.orchestra.run.trial093.stdout: auth debug = true 2026-02-22T21:54:17.484 INFO:teuthology.orchestra.run.trial093.stdout: 2026-02-22T21:54:17.484 INFO:teuthology.orchestra.run.trial093.stdout: ms die on old message = true 2026-02-22T21:54:17.484 INFO:teuthology.orchestra.run.trial093.stdout: ms die on bug = true 2026-02-22T21:54:17.485 INFO:teuthology.orchestra.run.trial093.stdout: 2026-02-22T21:54:17.485 INFO:teuthology.orchestra.run.trial093.stdout: mon max pg per osd = 10000 # >= luminous 2026-02-22T21:54:17.485 INFO:teuthology.orchestra.run.trial093.stdout: mon pg warn max object skew = 0 2026-02-22T21:54:17.485 INFO:teuthology.orchestra.run.trial093.stdout: 2026-02-22T21:54:17.485 INFO:teuthology.orchestra.run.trial093.stdout: # disable pg_autoscaler by default for new pools 2026-02-22T21:54:17.485 INFO:teuthology.orchestra.run.trial093.stdout: osd_pool_default_pg_autoscale_mode = off 2026-02-22T21:54:17.485 INFO:teuthology.orchestra.run.trial093.stdout: 2026-02-22T21:54:17.485 INFO:teuthology.orchestra.run.trial093.stdout: osd pool default size = 2 2026-02-22T21:54:17.485 INFO:teuthology.orchestra.run.trial093.stdout: 2026-02-22T21:54:17.485 INFO:teuthology.orchestra.run.trial093.stdout: mon osd allow primary affinity = true 2026-02-22T21:54:17.485 INFO:teuthology.orchestra.run.trial093.stdout: mon osd allow pg remap = true 2026-02-22T21:54:17.485 INFO:teuthology.orchestra.run.trial093.stdout: mon warn on legacy crush tunables = false 2026-02-22T21:54:17.486 INFO:teuthology.orchestra.run.trial093.stdout: mon warn on crush straw calc version zero = false 2026-02-22T21:54:17.486 INFO:teuthology.orchestra.run.trial093.stdout: mon warn on no sortbitwise = false 2026-02-22T21:54:17.486 INFO:teuthology.orchestra.run.trial093.stdout: mon warn on osd down out interval zero = false 2026-02-22T21:54:17.486 INFO:teuthology.orchestra.run.trial093.stdout: mon warn on too few osds = false 2026-02-22T21:54:17.486 INFO:teuthology.orchestra.run.trial093.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2026-02-22T21:54:17.486 INFO:teuthology.orchestra.run.trial093.stdout: mon_warn_on_pool_no_redundancy = false 2026-02-22T21:54:17.486 INFO:teuthology.orchestra.run.trial093.stdout: mon_allow_pool_size_one = true 2026-02-22T21:54:17.486 INFO:teuthology.orchestra.run.trial093.stdout: 2026-02-22T21:54:17.486 INFO:teuthology.orchestra.run.trial093.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2026-02-22T21:54:17.486 INFO:teuthology.orchestra.run.trial093.stdout: 2026-02-22T21:54:17.486 INFO:teuthology.orchestra.run.trial093.stdout: osd default data pool replay window = 5 2026-02-22T21:54:17.486 INFO:teuthology.orchestra.run.trial093.stdout: 2026-02-22T21:54:17.486 INFO:teuthology.orchestra.run.trial093.stdout: mon allow pool delete = true 2026-02-22T21:54:17.487 INFO:teuthology.orchestra.run.trial093.stdout: 2026-02-22T21:54:17.487 INFO:teuthology.orchestra.run.trial093.stdout: mon cluster log file level = debug 2026-02-22T21:54:17.487 INFO:teuthology.orchestra.run.trial093.stdout: debug asserts on shutdown = true 2026-02-22T21:54:17.487 INFO:teuthology.orchestra.run.trial093.stdout: mon health detail to clog = false 2026-02-22T21:54:17.487 INFO:teuthology.orchestra.run.trial093.stdout: mon host = 10.20.193.93 2026-02-22T21:54:17.487 INFO:teuthology.orchestra.run.trial093.stdout: mon warn on pool no app = False 2026-02-22T21:54:17.487 INFO:teuthology.orchestra.run.trial093.stdout: fsid = 63f41746-3baf-462b-9374-bd9179f37750 2026-02-22T21:54:17.487 INFO:teuthology.orchestra.run.trial093.stdout: 2026-02-22T21:54:17.487 INFO:teuthology.orchestra.run.trial093.stdout:[osd] 2026-02-22T21:54:17.487 INFO:teuthology.orchestra.run.trial093.stdout: osd journal size = 100 2026-02-22T21:54:17.487 INFO:teuthology.orchestra.run.trial093.stdout: 2026-02-22T21:54:17.488 INFO:teuthology.orchestra.run.trial093.stdout: osd scrub load threshold = 5.0 2026-02-22T21:54:17.488 INFO:teuthology.orchestra.run.trial093.stdout: osd scrub max interval = 600 2026-02-22T21:54:17.488 INFO:teuthology.orchestra.run.trial093.stdout: osd mclock profile = high_recovery_ops 2026-02-22T21:54:17.488 INFO:teuthology.orchestra.run.trial093.stdout: osd mclock skip benchmark = true 2026-02-22T21:54:17.488 INFO:teuthology.orchestra.run.trial093.stdout: 2026-02-22T21:54:17.488 INFO:teuthology.orchestra.run.trial093.stdout: osd recover clone overlap = true 2026-02-22T21:54:17.488 INFO:teuthology.orchestra.run.trial093.stdout: osd recovery max chunk = 1048576 2026-02-22T21:54:17.488 INFO:teuthology.orchestra.run.trial093.stdout: 2026-02-22T21:54:17.488 INFO:teuthology.orchestra.run.trial093.stdout: osd debug shutdown = true 2026-02-22T21:54:17.488 INFO:teuthology.orchestra.run.trial093.stdout: osd debug op order = true 2026-02-22T21:54:17.488 INFO:teuthology.orchestra.run.trial093.stdout: osd debug verify stray on activate = true 2026-02-22T21:54:17.488 INFO:teuthology.orchestra.run.trial093.stdout: osd debug trim objects = true 2026-02-22T21:54:17.488 INFO:teuthology.orchestra.run.trial093.stdout: 2026-02-22T21:54:17.489 INFO:teuthology.orchestra.run.trial093.stdout: osd open classes on start = true 2026-02-22T21:54:17.489 INFO:teuthology.orchestra.run.trial093.stdout: osd debug pg log writeout = true 2026-02-22T21:54:17.489 INFO:teuthology.orchestra.run.trial093.stdout: 2026-02-22T21:54:17.489 INFO:teuthology.orchestra.run.trial093.stdout: osd deep scrub update digest min age = 30 2026-02-22T21:54:17.489 INFO:teuthology.orchestra.run.trial093.stdout: 2026-02-22T21:54:17.489 INFO:teuthology.orchestra.run.trial093.stdout: osd map max advance = 10 2026-02-22T21:54:17.489 INFO:teuthology.orchestra.run.trial093.stdout: 2026-02-22T21:54:17.489 INFO:teuthology.orchestra.run.trial093.stdout: journal zero on create = true 2026-02-22T21:54:17.489 INFO:teuthology.orchestra.run.trial093.stdout: 2026-02-22T21:54:17.489 INFO:teuthology.orchestra.run.trial093.stdout: filestore ondisk finisher threads = 3 2026-02-22T21:54:17.489 INFO:teuthology.orchestra.run.trial093.stdout: filestore apply finisher threads = 3 2026-02-22T21:54:17.489 INFO:teuthology.orchestra.run.trial093.stdout: 2026-02-22T21:54:17.489 INFO:teuthology.orchestra.run.trial093.stdout: bdev debug aio = true 2026-02-22T21:54:17.490 INFO:teuthology.orchestra.run.trial093.stdout: osd debug misdirected ops = true 2026-02-22T21:54:17.490 INFO:teuthology.orchestra.run.trial093.stdout: bluefs allocator = hybrid 2026-02-22T21:54:17.490 INFO:teuthology.orchestra.run.trial093.stdout: bluestore allocator = hybrid 2026-02-22T21:54:17.490 INFO:teuthology.orchestra.run.trial093.stdout: bluestore block size = 96636764160 2026-02-22T21:54:17.490 INFO:teuthology.orchestra.run.trial093.stdout: bluestore fsck on mount = True 2026-02-22T21:54:17.490 INFO:teuthology.orchestra.run.trial093.stdout: debug bluefs = 1/20 2026-02-22T21:54:17.490 INFO:teuthology.orchestra.run.trial093.stdout: debug bluestore = 1/20 2026-02-22T21:54:17.490 INFO:teuthology.orchestra.run.trial093.stdout: debug ms = 1 2026-02-22T21:54:17.490 INFO:teuthology.orchestra.run.trial093.stdout: debug osd = 20 2026-02-22T21:54:17.490 INFO:teuthology.orchestra.run.trial093.stdout: debug rocksdb = 4/10 2026-02-22T21:54:17.490 INFO:teuthology.orchestra.run.trial093.stdout: mon osd backfillfull_ratio = 0.85 2026-02-22T21:54:17.490 INFO:teuthology.orchestra.run.trial093.stdout: mon osd full ratio = 0.9 2026-02-22T21:54:17.490 INFO:teuthology.orchestra.run.trial093.stdout: mon osd nearfull ratio = 0.8 2026-02-22T21:54:17.491 INFO:teuthology.orchestra.run.trial093.stdout: osd failsafe full ratio = 0.95 2026-02-22T21:54:17.491 INFO:teuthology.orchestra.run.trial093.stdout: osd objectstore = bluestore 2026-02-22T21:54:17.491 INFO:teuthology.orchestra.run.trial093.stdout: 2026-02-22T21:54:17.491 INFO:teuthology.orchestra.run.trial093.stdout:[mgr] 2026-02-22T21:54:17.491 INFO:teuthology.orchestra.run.trial093.stdout: debug ms = 1 2026-02-22T21:54:17.491 INFO:teuthology.orchestra.run.trial093.stdout: debug mgr = 20 2026-02-22T21:54:17.491 INFO:teuthology.orchestra.run.trial093.stdout: debug mon = 20 2026-02-22T21:54:17.491 INFO:teuthology.orchestra.run.trial093.stdout: debug auth = 20 2026-02-22T21:54:17.491 INFO:teuthology.orchestra.run.trial093.stdout: mon reweight min pgs per osd = 4 2026-02-22T21:54:17.491 INFO:teuthology.orchestra.run.trial093.stdout: mon reweight min bytes per osd = 10 2026-02-22T21:54:17.491 INFO:teuthology.orchestra.run.trial093.stdout: mgr/telemetry/nag = false 2026-02-22T21:54:17.491 INFO:teuthology.orchestra.run.trial093.stdout: 2026-02-22T21:54:17.492 INFO:teuthology.orchestra.run.trial093.stdout:[mon] 2026-02-22T21:54:17.492 INFO:teuthology.orchestra.run.trial093.stdout: debug ms = 1 2026-02-22T21:54:17.492 INFO:teuthology.orchestra.run.trial093.stdout: debug mon = 20 2026-02-22T21:54:17.492 INFO:teuthology.orchestra.run.trial093.stdout: debug paxos = 20 2026-02-22T21:54:17.492 INFO:teuthology.orchestra.run.trial093.stdout: debug auth = 20 2026-02-22T21:54:17.492 INFO:teuthology.orchestra.run.trial093.stdout: mon data avail warn = 5 2026-02-22T21:54:17.492 INFO:teuthology.orchestra.run.trial093.stdout: mon mgr mkfs grace = 240 2026-02-22T21:54:17.492 INFO:teuthology.orchestra.run.trial093.stdout: mon reweight min pgs per osd = 4 2026-02-22T21:54:17.492 INFO:teuthology.orchestra.run.trial093.stdout: mon osd reporter subtree level = osd 2026-02-22T21:54:17.492 INFO:teuthology.orchestra.run.trial093.stdout: mon osd prime pg temp = true 2026-02-22T21:54:17.492 INFO:teuthology.orchestra.run.trial093.stdout: mon reweight min bytes per osd = 10 2026-02-22T21:54:17.492 INFO:teuthology.orchestra.run.trial093.stdout: 2026-02-22T21:54:17.493 INFO:teuthology.orchestra.run.trial093.stdout: # rotate auth tickets quickly to exercise renewal paths 2026-02-22T21:54:17.493 INFO:teuthology.orchestra.run.trial093.stdout: auth mon ticket ttl = 660 # 11m 2026-02-22T21:54:17.493 INFO:teuthology.orchestra.run.trial093.stdout: auth service ticket ttl = 240 # 4m 2026-02-22T21:54:17.493 INFO:teuthology.orchestra.run.trial093.stdout: 2026-02-22T21:54:17.493 INFO:teuthology.orchestra.run.trial093.stdout: # don't complain about insecure global_id in the test suite 2026-02-22T21:54:17.493 INFO:teuthology.orchestra.run.trial093.stdout: mon_warn_on_insecure_global_id_reclaim = false 2026-02-22T21:54:17.493 INFO:teuthology.orchestra.run.trial093.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2026-02-22T21:54:17.493 INFO:teuthology.orchestra.run.trial093.stdout: 2026-02-22T21:54:17.493 INFO:teuthology.orchestra.run.trial093.stdout: # 1m isn't quite enough 2026-02-22T21:54:17.493 INFO:teuthology.orchestra.run.trial093.stdout: mon_down_mkfs_grace = 2m 2026-02-22T21:54:17.493 INFO:teuthology.orchestra.run.trial093.stdout: 2026-02-22T21:54:17.493 INFO:teuthology.orchestra.run.trial093.stdout: mon_warn_on_filestore_osds = false 2026-02-22T21:54:17.493 INFO:teuthology.orchestra.run.trial093.stdout: 2026-02-22T21:54:17.494 INFO:teuthology.orchestra.run.trial093.stdout:[client] 2026-02-22T21:54:17.494 INFO:teuthology.orchestra.run.trial093.stdout: rgw cache enabled = true 2026-02-22T21:54:17.494 INFO:teuthology.orchestra.run.trial093.stdout: rgw enable ops log = true 2026-02-22T21:54:17.494 INFO:teuthology.orchestra.run.trial093.stdout: rgw enable usage log = true 2026-02-22T21:54:17.494 INFO:teuthology.orchestra.run.trial093.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2026-02-22T21:54:17.494 INFO:teuthology.orchestra.run.trial093.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2026-02-22T21:54:17.494 INFO:teuthology.orchestra.run.trial093.stdout:[mon.a] 2026-02-22T21:54:17.495 INFO:tasks.ceph:Creating admin key on mon.a... 2026-02-22T21:54:17.495 DEBUG:teuthology.orchestra.run.trial093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2026-02-22T21:54:17.561 INFO:tasks.ceph:Copying monmap to all nodes... 2026-02-22T21:54:17.562 DEBUG:teuthology.orchestra.run.trial093:> set -ex 2026-02-22T21:54:17.562 DEBUG:teuthology.orchestra.run.trial093:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2026-02-22T21:54:17.606 DEBUG:teuthology.orchestra.run.trial093:> set -ex 2026-02-22T21:54:17.607 DEBUG:teuthology.orchestra.run.trial093:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2026-02-22T21:54:17.652 INFO:tasks.ceph:Sending monmap to node ubuntu@trial093.front.sepia.ceph.com 2026-02-22T21:54:17.652 DEBUG:teuthology.orchestra.run.trial093:> set -ex 2026-02-22T21:54:17.652 DEBUG:teuthology.orchestra.run.trial093:> sudo dd of=/etc/ceph/ceph.keyring 2026-02-22T21:54:17.652 DEBUG:teuthology.orchestra.run.trial093:> sudo chmod 0644 /etc/ceph/ceph.keyring 2026-02-22T21:54:17.708 DEBUG:teuthology.orchestra.run.trial093:> set -ex 2026-02-22T21:54:17.708 DEBUG:teuthology.orchestra.run.trial093:> dd of=/home/ubuntu/cephtest/ceph.monmap 2026-02-22T21:54:17.760 INFO:tasks.ceph:Setting up mon nodes... 2026-02-22T21:54:17.760 INFO:tasks.ceph:Setting up mgr nodes... 2026-02-22T21:54:17.760 DEBUG:teuthology.orchestra.run.trial093:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2026-02-22T21:54:17.824 INFO:teuthology.orchestra.run.trial093.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2026-02-22T21:54:17.827 INFO:tasks.ceph:Setting up mds nodes... 2026-02-22T21:54:17.827 INFO:tasks.ceph_client:Setting up client nodes... 2026-02-22T21:54:17.827 DEBUG:teuthology.orchestra.run.trial093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2026-02-22T21:54:17.883 INFO:teuthology.orchestra.run.trial093.stdout:creating /etc/ceph/ceph.client.0.keyring 2026-02-22T21:54:17.891 INFO:tasks.ceph:Running mkfs on osd nodes... 2026-02-22T21:54:17.891 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@trial093.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2'}} 2026-02-22T21:54:17.891 DEBUG:teuthology.orchestra.run.trial093:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2026-02-22T21:54:17.939 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2'} 2026-02-22T21:54:17.940 INFO:tasks.ceph:role: osd.0 2026-02-22T21:54:17.940 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@trial093.front.sepia.ceph.com 2026-02-22T21:54:17.940 DEBUG:teuthology.orchestra.run.trial093:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2026-02-22T21:54:17.993 INFO:teuthology.orchestra.run.trial093.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=45875200 blks 2026-02-22T21:54:17.993 INFO:teuthology.orchestra.run.trial093.stdout: = sectsz=4096 attr=2, projid32bit=1 2026-02-22T21:54:17.993 INFO:teuthology.orchestra.run.trial093.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2026-02-22T21:54:17.993 INFO:teuthology.orchestra.run.trial093.stdout: = reflink=1 bigtime=0 inobtcount=0 2026-02-22T21:54:17.994 INFO:teuthology.orchestra.run.trial093.stdout:data = bsize=4096 blocks=183500800, imaxpct=25 2026-02-22T21:54:17.994 INFO:teuthology.orchestra.run.trial093.stdout: = sunit=0 swidth=0 blks 2026-02-22T21:54:17.994 INFO:teuthology.orchestra.run.trial093.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2026-02-22T21:54:17.994 INFO:teuthology.orchestra.run.trial093.stdout:log =internal log bsize=4096 blocks=89600, version=2 2026-02-22T21:54:17.994 INFO:teuthology.orchestra.run.trial093.stdout: = sectsz=4096 sunit=1 blks, lazy-count=1 2026-02-22T21:54:17.994 INFO:teuthology.orchestra.run.trial093.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2026-02-22T21:54:18.608 INFO:teuthology.orchestra.run.trial093.stdout:Discarding blocks...Done. 2026-02-22T21:54:18.610 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@trial093.front.sepia.ceph.com -o noatime 2026-02-22T21:54:18.610 DEBUG:teuthology.orchestra.run.trial093:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2026-02-22T21:54:18.665 DEBUG:teuthology.orchestra.run.trial093:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2026-02-22T21:54:18.713 INFO:teuthology.orchestra.run.trial093.stderr:sudo: /sbin/restorecon: command not found 2026-02-22T21:54:18.714 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-22T21:54:18.715 DEBUG:teuthology.orchestra.run.trial093:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2026-02-22T21:54:18.763 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2'} 2026-02-22T21:54:18.763 INFO:tasks.ceph:role: osd.1 2026-02-22T21:54:18.764 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@trial093.front.sepia.ceph.com 2026-02-22T21:54:18.764 DEBUG:teuthology.orchestra.run.trial093:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2026-02-22T21:54:18.813 INFO:teuthology.orchestra.run.trial093.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=45875200 blks 2026-02-22T21:54:18.813 INFO:teuthology.orchestra.run.trial093.stdout: = sectsz=4096 attr=2, projid32bit=1 2026-02-22T21:54:18.813 INFO:teuthology.orchestra.run.trial093.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2026-02-22T21:54:18.814 INFO:teuthology.orchestra.run.trial093.stdout: = reflink=1 bigtime=0 inobtcount=0 2026-02-22T21:54:18.814 INFO:teuthology.orchestra.run.trial093.stdout:data = bsize=4096 blocks=183500800, imaxpct=25 2026-02-22T21:54:18.814 INFO:teuthology.orchestra.run.trial093.stdout: = sunit=0 swidth=0 blks 2026-02-22T21:54:18.814 INFO:teuthology.orchestra.run.trial093.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2026-02-22T21:54:18.814 INFO:teuthology.orchestra.run.trial093.stdout:log =internal log bsize=4096 blocks=89600, version=2 2026-02-22T21:54:18.814 INFO:teuthology.orchestra.run.trial093.stdout: = sectsz=4096 sunit=1 blks, lazy-count=1 2026-02-22T21:54:18.814 INFO:teuthology.orchestra.run.trial093.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2026-02-22T21:54:19.381 INFO:teuthology.orchestra.run.trial093.stdout:Discarding blocks...Done. 2026-02-22T21:54:19.383 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@trial093.front.sepia.ceph.com -o noatime 2026-02-22T21:54:19.383 DEBUG:teuthology.orchestra.run.trial093:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2026-02-22T21:54:19.440 DEBUG:teuthology.orchestra.run.trial093:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2026-02-22T21:54:19.489 INFO:teuthology.orchestra.run.trial093.stderr:sudo: /sbin/restorecon: command not found 2026-02-22T21:54:19.489 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-22T21:54:19.489 DEBUG:teuthology.orchestra.run.trial093:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2026-02-22T21:54:19.553 INFO:teuthology.orchestra.run.trial093.stderr:2026-02-22T21:54:19.547+0000 7f0556e318c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2026-02-22T21:54:19.553 INFO:teuthology.orchestra.run.trial093.stderr:2026-02-22T21:54:19.547+0000 7f0556e318c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2026-02-22T21:54:19.554 INFO:teuthology.orchestra.run.trial093.stderr:2026-02-22T21:54:19.547+0000 7f0556e318c0 -1 bdev(0x5593ae609800 /var/lib/ceph/osd/ceph-0/block) open stat got: (1) Operation not permitted 2026-02-22T21:54:19.554 INFO:teuthology.orchestra.run.trial093.stderr:2026-02-22T21:54:19.547+0000 7f0556e318c0 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2026-02-22T21:54:22.654 DEBUG:teuthology.orchestra.run.trial093:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2026-02-22T21:54:22.704 DEBUG:teuthology.orchestra.run.trial093:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2026-02-22T21:54:22.768 INFO:teuthology.orchestra.run.trial093.stderr:2026-02-22T21:54:22.762+0000 7ff4a5fdc8c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2026-02-22T21:54:22.769 INFO:teuthology.orchestra.run.trial093.stderr:2026-02-22T21:54:22.762+0000 7ff4a5fdc8c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2026-02-22T21:54:22.769 INFO:teuthology.orchestra.run.trial093.stderr:2026-02-22T21:54:22.762+0000 7ff4a5fdc8c0 -1 bdev(0x556772c45800 /var/lib/ceph/osd/ceph-1/block) open stat got: (1) Operation not permitted 2026-02-22T21:54:22.769 INFO:teuthology.orchestra.run.trial093.stderr:2026-02-22T21:54:22.762+0000 7ff4a5fdc8c0 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2026-02-22T21:54:25.875 DEBUG:teuthology.orchestra.run.trial093:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2026-02-22T21:54:25.928 INFO:tasks.ceph:Reading keys from all nodes... 2026-02-22T21:54:25.929 DEBUG:teuthology.orchestra.run.trial093:> set -ex 2026-02-22T21:54:25.929 DEBUG:teuthology.orchestra.run.trial093:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2026-02-22T21:54:25.981 DEBUG:teuthology.orchestra.run.trial093:> set -ex 2026-02-22T21:54:25.982 DEBUG:teuthology.orchestra.run.trial093:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2026-02-22T21:54:26.033 DEBUG:teuthology.orchestra.run.trial093:> set -ex 2026-02-22T21:54:26.033 DEBUG:teuthology.orchestra.run.trial093:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2026-02-22T21:54:26.085 DEBUG:teuthology.orchestra.run.trial093:> set -ex 2026-02-22T21:54:26.085 DEBUG:teuthology.orchestra.run.trial093:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2026-02-22T21:54:26.132 INFO:tasks.ceph:Adding keys to all mons... 2026-02-22T21:54:26.132 DEBUG:teuthology.orchestra.run.trial093:> sudo tee -a /etc/ceph/ceph.keyring 2026-02-22T21:54:26.182 INFO:teuthology.orchestra.run.trial093.stdout:[mgr.x] 2026-02-22T21:54:26.182 INFO:teuthology.orchestra.run.trial093.stdout: key = AQAJe5tptkMZMRAAdVxRc52NB0d4lNT7Lpd1LQ== 2026-02-22T21:54:26.182 INFO:teuthology.orchestra.run.trial093.stdout:[osd.0] 2026-02-22T21:54:26.182 INFO:teuthology.orchestra.run.trial093.stdout: key = AQALe5tpSJnpIBAAjDw+KzAJhRWPKLJE+PUOxA== 2026-02-22T21:54:26.182 INFO:teuthology.orchestra.run.trial093.stdout:[osd.1] 2026-02-22T21:54:26.183 INFO:teuthology.orchestra.run.trial093.stdout: key = AQAOe5tprNe9LRAAPO8azSqKTjLJWu015WS1sg== 2026-02-22T21:54:26.183 INFO:teuthology.orchestra.run.trial093.stdout:[client.0] 2026-02-22T21:54:26.183 INFO:teuthology.orchestra.run.trial093.stdout: key = AQAJe5tpo+OgNBAAj5nE2zdzUDuIJnSTAxrpEw== 2026-02-22T21:54:26.184 DEBUG:teuthology.orchestra.run.trial093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2026-02-22T21:54:26.249 DEBUG:teuthology.orchestra.run.trial093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-22T21:54:26.312 DEBUG:teuthology.orchestra.run.trial093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-22T21:54:26.376 DEBUG:teuthology.orchestra.run.trial093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2026-02-22T21:54:26.440 INFO:tasks.ceph:Running mkfs on mon nodes... 2026-02-22T21:54:26.441 DEBUG:teuthology.orchestra.run.trial093:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2026-02-22T21:54:26.491 DEBUG:teuthology.orchestra.run.trial093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2026-02-22T21:54:26.561 DEBUG:teuthology.orchestra.run.trial093:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2026-02-22T21:54:26.612 DEBUG:teuthology.orchestra.run.trial093:> rm -- /home/ubuntu/cephtest/ceph.monmap 2026-02-22T21:54:26.659 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2026-02-22T21:54:26.660 INFO:tasks.ceph.mon.a:Restarting daemon 2026-02-22T21:54:26.660 DEBUG:teuthology.orchestra.run.trial093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2026-02-22T21:54:26.701 INFO:tasks.ceph.mon.a:Started 2026-02-22T21:54:26.701 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2026-02-22T21:54:26.701 INFO:tasks.ceph.mgr.x:Restarting daemon 2026-02-22T21:54:26.701 DEBUG:teuthology.orchestra.run.trial093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2026-02-22T21:54:26.703 INFO:tasks.ceph.mgr.x:Started 2026-02-22T21:54:26.704 INFO:tasks.ceph:Setting crush tunables to default 2026-02-22T21:54:26.704 DEBUG:teuthology.orchestra.run.trial093:> sudo ceph --cluster ceph osd crush tunables default 2026-02-22T21:54:26.825 INFO:teuthology.orchestra.run.trial093.stderr:adjusted tunables profile to default 2026-02-22T21:54:26.835 INFO:tasks.ceph:check_enable_crimson: False 2026-02-22T21:54:26.835 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2026-02-22T21:54:26.835 DEBUG:teuthology.orchestra.run.trial093:> set -ex 2026-02-22T21:54:26.835 DEBUG:teuthology.orchestra.run.trial093:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2026-02-22T21:54:26.845 DEBUG:teuthology.orchestra.run.trial093:> set -ex 2026-02-22T21:54:26.845 DEBUG:teuthology.orchestra.run.trial093:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2026-02-22T21:54:26.897 DEBUG:teuthology.orchestra.run.trial093:> sudo ceph --cluster ceph osd new 7161d54e-1b56-4091-a2ca-0b42ae8b3475 0 2026-02-22T21:54:27.018 INFO:tasks.ceph.mgr.x.trial093.stderr:2026-02-22T21:54:27.014+0000 7f4664904280 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2026-02-22T21:54:27.038 INFO:teuthology.orchestra.run.trial093.stdout:0 2026-02-22T21:54:27.044 INFO:tasks.ceph.mgr.x.trial093.stderr:2026-02-22T21:54:27.042+0000 7f4664904280 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2026-02-22T21:54:27.048 DEBUG:teuthology.orchestra.run.trial093:> sudo ceph --cluster ceph osd new a7ea7aed-6c59-4184-8da7-354423cced23 1 2026-02-22T21:54:27.081 INFO:tasks.ceph.mgr.x.trial093.stderr:2026-02-22T21:54:27.078+0000 7f4664904280 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2026-02-22T21:54:27.147 INFO:teuthology.orchestra.run.trial093.stdout:1 2026-02-22T21:54:27.156 INFO:tasks.ceph.osd.0:Restarting daemon 2026-02-22T21:54:27.157 DEBUG:teuthology.orchestra.run.trial093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2026-02-22T21:54:27.168 INFO:tasks.ceph.mgr.x.trial093.stderr:2026-02-22T21:54:27.166+0000 7f4664904280 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2026-02-22T21:54:27.169 INFO:tasks.ceph.osd.0:Started 2026-02-22T21:54:27.169 INFO:tasks.ceph.osd.1:Restarting daemon 2026-02-22T21:54:27.170 DEBUG:teuthology.orchestra.run.trial093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2026-02-22T21:54:27.172 INFO:tasks.ceph.osd.1:Started 2026-02-22T21:54:27.172 DEBUG:teuthology.orchestra.run.trial093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2026-02-22T21:54:27.249 INFO:tasks.ceph.mgr.x.trial093.stderr:2026-02-22T21:54:27.246+0000 7f4664904280 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2026-02-22T21:54:27.267 INFO:teuthology.orchestra.run.trial093.stdout: 2026-02-22T21:54:27.267 INFO:teuthology.orchestra.run.trial093.stdout:{"epoch":4,"fsid":"63f41746-3baf-462b-9374-bd9179f37750","created":"2026-02-22T21:54:26.784109+0000","modified":"2026-02-22T21:54:27.145522+0000","last_up_change":"0.000000","last_in_change":"2026-02-22T21:54:27.145522+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":2,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"7161d54e-1b56-4091-a2ca-0b42ae8b3475","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"a7ea7aed-6c59-4184-8da7-354423cced23","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2026-02-22T21:54:27.277 INFO:tasks.ceph.ceph_manager.ceph:[] 2026-02-22T21:54:27.277 INFO:tasks.ceph:Waiting for OSDs to come up 2026-02-22T21:54:27.286 INFO:tasks.ceph.mgr.x.trial093.stderr:2026-02-22T21:54:27.282+0000 7f4664904280 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2026-02-22T21:54:27.362 INFO:tasks.ceph.mgr.x.trial093.stderr:2026-02-22T21:54:27.358+0000 7f4664904280 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2026-02-22T21:54:27.479 DEBUG:teuthology.orchestra.run.trial093:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2026-02-22T21:54:27.535 INFO:tasks.ceph.mgr.x.trial093.stderr:2026-02-22T21:54:27.530+0000 7f4664904280 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2026-02-22T21:54:27.569 INFO:teuthology.misc.health.trial093.stdout: 2026-02-22T21:54:27.570 INFO:teuthology.misc.health.trial093.stdout:{"epoch":4,"fsid":"63f41746-3baf-462b-9374-bd9179f37750","created":"2026-02-22T21:54:26.784109+0000","modified":"2026-02-22T21:54:27.145522+0000","last_up_change":"0.000000","last_in_change":"2026-02-22T21:54:27.145522+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":2,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"7161d54e-1b56-4091-a2ca-0b42ae8b3475","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"a7ea7aed-6c59-4184-8da7-354423cced23","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2026-02-22T21:54:27.578 DEBUG:teuthology.misc:0 of 2 OSDs are up 2026-02-22T21:54:27.843 INFO:tasks.ceph.mgr.x.trial093.stderr:2026-02-22T21:54:27.838+0000 7f4664904280 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2026-02-22T21:54:27.938 INFO:tasks.ceph.mgr.x.trial093.stderr:2026-02-22T21:54:27.934+0000 7f4664904280 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2026-02-22T21:54:27.966 INFO:tasks.ceph.mgr.x.trial093.stderr:2026-02-22T21:54:27.962+0000 7f4664904280 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2026-02-22T21:54:27.999 INFO:tasks.ceph.mgr.x.trial093.stderr:2026-02-22T21:54:27.994+0000 7f4664904280 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2026-02-22T21:54:28.042 INFO:tasks.ceph.mgr.x.trial093.stderr:2026-02-22T21:54:28.038+0000 7f4664904280 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2026-02-22T21:54:28.145 INFO:tasks.ceph.mgr.x.trial093.stderr:2026-02-22T21:54:28.142+0000 7f4664904280 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2026-02-22T21:54:28.344 INFO:tasks.ceph.mgr.x.trial093.stderr:ERROR:root:Module 'xmltodict' is not installed. 2026-02-22T21:54:28.521 INFO:tasks.ceph.mgr.x.trial093.stderr:2026-02-22T21:54:28.518+0000 7f4664904280 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2026-02-22T21:54:28.553 INFO:tasks.ceph.mgr.x.trial093.stderr:2026-02-22T21:54:28.550+0000 7f4664904280 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2026-02-22T21:54:28.649 INFO:tasks.ceph.mgr.x.trial093.stderr:2026-02-22T21:54:28.646+0000 7f4664904280 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2026-02-22T21:54:28.674 INFO:tasks.ceph.mgr.x.trial093.stderr:2026-02-22T21:54:28.670+0000 7f4664904280 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2026-02-22T21:54:28.701 INFO:tasks.ceph.mgr.x.trial093.stderr:2026-02-22T21:54:28.698+0000 7f4664904280 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2026-02-22T21:54:28.729 INFO:tasks.ceph.mgr.x.trial093.stderr:2026-02-22T21:54:28.726+0000 7f4664904280 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2026-02-22T21:54:28.769 INFO:tasks.ceph.osd.0.trial093.stderr:2026-02-22T21:54:28.766+0000 7f17f7ec88c0 -1 Falling back to public interface 2026-02-22T21:54:28.808 INFO:tasks.ceph.osd.1.trial093.stderr:2026-02-22T21:54:28.806+0000 7f0c996418c0 -1 Falling back to public interface 2026-02-22T21:54:28.852 INFO:tasks.ceph.mgr.x.trial093.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2026-02-22T21:54:28.852 INFO:tasks.ceph.mgr.x.trial093.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2026-02-22T21:54:28.852 INFO:tasks.ceph.mgr.x.trial093.stderr: from numpy import show_config as show_numpy_config 2026-02-22T21:54:28.859 INFO:tasks.ceph.mgr.x.trial093.stderr:2026-02-22T21:54:28.854+0000 7f4664904280 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2026-02-22T21:54:28.886 INFO:tasks.ceph.mgr.x.trial093.stderr:2026-02-22T21:54:28.882+0000 7f4664904280 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2026-02-22T21:54:28.926 INFO:tasks.ceph.mgr.x.trial093.stderr:2026-02-22T21:54:28.922+0000 7f4664904280 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2026-02-22T21:54:29.950 INFO:tasks.ceph.mgr.x.trial093.stderr:2026-02-22T21:54:29.946+0000 7f4583151640 -1 mgr.server handle_report got status from non-daemon mon.a 2026-02-22T21:54:32.434 INFO:tasks.ceph.osd.1.trial093.stderr:2026-02-22T21:54:32.430+0000 7f0c996418c0 -1 osd.1 0 log_to_monitors true 2026-02-22T21:54:32.655 INFO:tasks.ceph.osd.0.trial093.stderr:2026-02-22T21:54:32.650+0000 7f17f7ec88c0 -1 osd.0 0 log_to_monitors true 2026-02-22T21:54:33.780 DEBUG:teuthology.orchestra.run.trial093:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2026-02-22T21:54:33.926 INFO:teuthology.misc.health.trial093.stdout: 2026-02-22T21:54:33.926 INFO:teuthology.misc.health.trial093.stdout:{"epoch":5,"fsid":"63f41746-3baf-462b-9374-bd9179f37750","created":"2026-02-22T21:54:26.784109+0000","modified":"2026-02-22T21:54:32.948184+0000","last_up_change":"0.000000","last_in_change":"2026-02-22T21:54:27.145522+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":3,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":2,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"7161d54e-1b56-4091-a2ca-0b42ae8b3475","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"a7ea7aed-6c59-4184-8da7-354423cced23","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2026-02-22T21:54:33.936 DEBUG:teuthology.misc:0 of 2 OSDs are up 2026-02-22T21:54:33.953 INFO:tasks.ceph.osd.1.trial093.stderr:2026-02-22T21:54:33.950+0000 7f0c955ea640 -1 osd.1 0 waiting for initial osdmap 2026-02-22T21:54:33.953 INFO:tasks.ceph.osd.0.trial093.stderr:2026-02-22T21:54:33.950+0000 7f17f3e71640 -1 osd.0 0 waiting for initial osdmap 2026-02-22T21:54:33.976 INFO:tasks.ceph.osd.1.trial093.stderr:2026-02-22T21:54:33.974+0000 7f0c8d41d640 -1 osd.1 6 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2026-02-22T21:54:33.988 INFO:tasks.ceph.osd.0.trial093.stderr:2026-02-22T21:54:33.986+0000 7f17ebca4640 -1 osd.0 6 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2026-02-22T21:54:40.138 DEBUG:teuthology.orchestra.run.trial093:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2026-02-22T21:54:40.280 INFO:teuthology.misc.health.trial093.stdout: 2026-02-22T21:54:40.280 INFO:teuthology.misc.health.trial093.stdout:{"epoch":10,"fsid":"63f41746-3baf-462b-9374-bd9179f37750","created":"2026-02-22T21:54:26.784109+0000","modified":"2026-02-22T21:54:37.959473+0000","last_up_change":"2026-02-22T21:54:34.951859+0000","last_in_change":"2026-02-22T21:54:27.145522+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":2,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2026-02-22T21:54:34.979937+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"10","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7161d54e-1b56-4091-a2ca-0b42ae8b3475","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":7,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.93:6800","nonce":1805491287},{"type":"v1","addr":"10.20.193.93:6801","nonce":1805491287}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.93:6802","nonce":1805491287},{"type":"v1","addr":"10.20.193.93:6803","nonce":1805491287}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.93:6806","nonce":1805491287},{"type":"v1","addr":"10.20.193.93:6807","nonce":1805491287}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.93:6804","nonce":1805491287},{"type":"v1","addr":"10.20.193.93:6805","nonce":1805491287}]},"public_addr":"10.20.193.93:6801/1805491287","cluster_addr":"10.20.193.93:6803/1805491287","heartbeat_back_addr":"10.20.193.93:6807/1805491287","heartbeat_front_addr":"10.20.193.93:6805/1805491287","state":["exists","up"]},{"osd":1,"uuid":"a7ea7aed-6c59-4184-8da7-354423cced23","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":7,"up_thru":8,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.93:6808","nonce":1536829300},{"type":"v1","addr":"10.20.193.93:6809","nonce":1536829300}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.93:6810","nonce":1536829300},{"type":"v1","addr":"10.20.193.93:6811","nonce":1536829300}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.93:6814","nonce":1536829300},{"type":"v1","addr":"10.20.193.93:6815","nonce":1536829300}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.93:6812","nonce":1536829300},{"type":"v1","addr":"10.20.193.93:6813","nonce":1536829300}]},"public_addr":"10.20.193.93:6809/1536829300","cluster_addr":"10.20.193.93:6811/1536829300","heartbeat_back_addr":"10.20.193.93:6815/1536829300","heartbeat_front_addr":"10.20.193.93:6813/1536829300","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2026-02-22T21:54:40.289 DEBUG:teuthology.misc:2 of 2 OSDs are up 2026-02-22T21:54:40.289 INFO:tasks.ceph:Creating RBD pool 2026-02-22T21:54:40.289 DEBUG:teuthology.orchestra.run.trial093:> sudo ceph --cluster ceph osd pool create rbd 8 2026-02-22T21:54:40.971 INFO:teuthology.orchestra.run.trial093.stderr:pool 'rbd' created 2026-02-22T21:54:40.984 DEBUG:teuthology.orchestra.run.trial093:> rbd --cluster ceph pool init rbd 2026-02-22T21:54:43.984 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2026-02-22T21:54:43.984 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2026-02-22T21:54:43.984 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2026-02-22T21:54:43.985 DEBUG:teuthology.orchestra.run.trial093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2026-02-22T21:54:43.985 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2026-02-22T21:54:44.148 INFO:teuthology.orchestra.run.trial093.stdout: 2026-02-22T21:54:44.157 INFO:teuthology.orchestra.run.trial093.stdout:{"epoch":5,"flags":0,"active_gid":4101,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.93:6816","nonce":2593877255},{"type":"v1","addr":"10.20.193.93:6817","nonce":2593877255}]},"active_addr":"10.20.193.93:6817/2593877255","active_change":"2026-02-22T21:54:28.943616+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_pg_upmap_activity":{"name":"update_pg_upmap_activity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Updates pg_upmap activity stats to be used in `balancer status detail`","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:10.4.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.7.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.51.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"quay.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_count_max":{"name":"ssh_keepalive_count_max","type":"int","level":"advanced","flags":0,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"How many times ssh connections can fail liveness checks before the host is marked offline","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_interval":{"name":"ssh_keepalive_interval","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"How often ssh connections are checked for liveness","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_requests":{"name":"max_requests","type":"int","level":"advanced","flags":0,"default_value":"500","min":"","max":"","enum_allowed":[],"desc":"Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number. if un-finished request is removed, error message will be logged in the ceph-mgr log.","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"force_disabled_modules":{},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"10.20.193.93:0","nonce":174064999}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"10.20.193.93:0","nonce":2727633243}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"10.20.193.93:0","nonce":109117872}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"10.20.193.93:0","nonce":1903164185}]}]} 2026-02-22T21:54:44.161 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2026-02-22T21:54:44.161 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2026-02-22T21:54:44.161 DEBUG:teuthology.orchestra.run.trial093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2026-02-22T21:54:44.313 INFO:teuthology.orchestra.run.trial093.stdout: 2026-02-22T21:54:44.314 INFO:teuthology.orchestra.run.trial093.stdout:{"epoch":14,"fsid":"63f41746-3baf-462b-9374-bd9179f37750","created":"2026-02-22T21:54:26.784109+0000","modified":"2026-02-22T21:54:43.974269+0000","last_up_change":"2026-02-22T21:54:34.951859+0000","last_in_change":"2026-02-22T21:54:27.145522+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":2,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2026-02-22T21:54:34.979937+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"10","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2026-02-22T21:54:40.438651+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"14","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":14,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7161d54e-1b56-4091-a2ca-0b42ae8b3475","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":7,"up_thru":11,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.93:6800","nonce":1805491287},{"type":"v1","addr":"10.20.193.93:6801","nonce":1805491287}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.93:6802","nonce":1805491287},{"type":"v1","addr":"10.20.193.93:6803","nonce":1805491287}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.93:6806","nonce":1805491287},{"type":"v1","addr":"10.20.193.93:6807","nonce":1805491287}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.93:6804","nonce":1805491287},{"type":"v1","addr":"10.20.193.93:6805","nonce":1805491287}]},"public_addr":"10.20.193.93:6801/1805491287","cluster_addr":"10.20.193.93:6803/1805491287","heartbeat_back_addr":"10.20.193.93:6807/1805491287","heartbeat_front_addr":"10.20.193.93:6805/1805491287","state":["exists","up"]},{"osd":1,"uuid":"a7ea7aed-6c59-4184-8da7-354423cced23","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":7,"up_thru":11,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.93:6808","nonce":1536829300},{"type":"v1","addr":"10.20.193.93:6809","nonce":1536829300}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.93:6810","nonce":1536829300},{"type":"v1","addr":"10.20.193.93:6811","nonce":1536829300}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.93:6814","nonce":1536829300},{"type":"v1","addr":"10.20.193.93:6815","nonce":1536829300}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.93:6812","nonce":1536829300},{"type":"v1","addr":"10.20.193.93:6813","nonce":1536829300}]},"public_addr":"10.20.193.93:6809/1536829300","cluster_addr":"10.20.193.93:6811/1536829300","heartbeat_back_addr":"10.20.193.93:6815/1536829300","heartbeat_front_addr":"10.20.193.93:6813/1536829300","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_removed_snaps":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2026-02-22T21:54:44.324 INFO:tasks.ceph.ceph_manager.ceph:all up! 2026-02-22T21:54:44.324 DEBUG:teuthology.orchestra.run.trial093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2026-02-22T21:54:44.482 INFO:teuthology.orchestra.run.trial093.stdout: 2026-02-22T21:54:44.482 INFO:teuthology.orchestra.run.trial093.stdout:{"epoch":14,"fsid":"63f41746-3baf-462b-9374-bd9179f37750","created":"2026-02-22T21:54:26.784109+0000","modified":"2026-02-22T21:54:43.974269+0000","last_up_change":"2026-02-22T21:54:34.951859+0000","last_in_change":"2026-02-22T21:54:27.145522+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":2,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2026-02-22T21:54:34.979937+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"10","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2026-02-22T21:54:40.438651+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"14","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":14,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7161d54e-1b56-4091-a2ca-0b42ae8b3475","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":7,"up_thru":11,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.93:6800","nonce":1805491287},{"type":"v1","addr":"10.20.193.93:6801","nonce":1805491287}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.93:6802","nonce":1805491287},{"type":"v1","addr":"10.20.193.93:6803","nonce":1805491287}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.93:6806","nonce":1805491287},{"type":"v1","addr":"10.20.193.93:6807","nonce":1805491287}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.93:6804","nonce":1805491287},{"type":"v1","addr":"10.20.193.93:6805","nonce":1805491287}]},"public_addr":"10.20.193.93:6801/1805491287","cluster_addr":"10.20.193.93:6803/1805491287","heartbeat_back_addr":"10.20.193.93:6807/1805491287","heartbeat_front_addr":"10.20.193.93:6805/1805491287","state":["exists","up"]},{"osd":1,"uuid":"a7ea7aed-6c59-4184-8da7-354423cced23","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":7,"up_thru":11,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.93:6808","nonce":1536829300},{"type":"v1","addr":"10.20.193.93:6809","nonce":1536829300}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.93:6810","nonce":1536829300},{"type":"v1","addr":"10.20.193.93:6811","nonce":1536829300}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.93:6814","nonce":1536829300},{"type":"v1","addr":"10.20.193.93:6815","nonce":1536829300}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.93:6812","nonce":1536829300},{"type":"v1","addr":"10.20.193.93:6813","nonce":1536829300}]},"public_addr":"10.20.193.93:6809/1536829300","cluster_addr":"10.20.193.93:6811/1536829300","heartbeat_back_addr":"10.20.193.93:6815/1536829300","heartbeat_front_addr":"10.20.193.93:6813/1536829300","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_removed_snaps":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2026-02-22T21:54:44.493 DEBUG:teuthology.orchestra.run.trial093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2026-02-22T21:54:44.494 DEBUG:teuthology.orchestra.run.trial093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2026-02-22T21:54:44.576 INFO:teuthology.orchestra.run.trial093.stdout:30064771076 2026-02-22T21:54:44.577 DEBUG:teuthology.orchestra.run.trial093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2026-02-22T21:54:44.579 INFO:teuthology.orchestra.run.trial093.stdout:30064771077 2026-02-22T21:54:44.580 DEBUG:teuthology.orchestra.run.trial093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2026-02-22T21:54:44.768 INFO:teuthology.orchestra.run.trial093.stdout:30064771076 2026-02-22T21:54:44.770 INFO:teuthology.orchestra.run.trial093.stdout:30064771075 2026-02-22T21:54:44.778 INFO:tasks.ceph.ceph_manager.ceph:need seq 30064771077 got 30064771076 for osd.1 2026-02-22T21:54:44.782 INFO:tasks.ceph.ceph_manager.ceph:need seq 30064771076 got 30064771075 for osd.0 2026-02-22T21:54:45.779 DEBUG:teuthology.orchestra.run.trial093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2026-02-22T21:54:45.784 DEBUG:teuthology.orchestra.run.trial093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2026-02-22T21:54:45.937 INFO:teuthology.orchestra.run.trial093.stdout:30064771077 2026-02-22T21:54:45.949 INFO:tasks.ceph.ceph_manager.ceph:need seq 30064771077 got 30064771077 for osd.1 2026-02-22T21:54:45.949 DEBUG:teuthology.parallel:result is None 2026-02-22T21:54:45.970 INFO:teuthology.orchestra.run.trial093.stdout:30064771076 2026-02-22T21:54:45.981 INFO:tasks.ceph.ceph_manager.ceph:need seq 30064771076 got 30064771076 for osd.0 2026-02-22T21:54:45.982 DEBUG:teuthology.parallel:result is None 2026-02-22T21:54:45.982 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2026-02-22T21:54:45.982 DEBUG:teuthology.orchestra.run.trial093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T21:54:46.167 INFO:teuthology.orchestra.run.trial093.stdout: 2026-02-22T21:54:46.167 INFO:teuthology.orchestra.run.trial093.stderr:dumped all 2026-02-22T21:54:46.177 INFO:teuthology.orchestra.run.trial093.stdout:{"pg_ready":true,"pg_map":{"version":18,"stamp":"2026-02-22T21:54:44.948939+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":56,"num_read_kb":46,"num_write":81,"num_write_kb":1108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":55,"ondisk_log_size":55,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":55336,"kb_used_data":1400,"kb_used_omap":3,"kb_used_meta":53884,"kb_avail":188688344,"statfs":{"total":193273528320,"available":193216864256,"internally_reserved":0,"allocated":1433600,"data_stored":1243105,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":3188,"internal_metadata":55178124},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"3.978694"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":19,"reported_epoch":14,"state":"active+clean","last_fresh":"2026-02-22T21:54:43.978642+0000","last_change":"2026-02-22T21:54:43.978642+0000","last_active":"2026-02-22T21:54:43.978642+0000","last_peered":"2026-02-22T21:54:43.978642+0000","last_clean":"2026-02-22T21:54:43.978642+0000","last_became_active":"2026-02-22T21:54:41.977824+0000","last_became_peered":"2026-02-22T21:54:41.977824+0000","last_unstale":"2026-02-22T21:54:43.978642+0000","last_undegraded":"2026-02-22T21:54:43.978642+0000","last_fullsized":"2026-02-22T21:54:43.978642+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T21:54:40.966951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T21:54:40.966951+0000","last_clean_scrub_stamp":"2026-02-22T21:54:40.966951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:31:00.749250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00078258400000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":19,"reported_epoch":14,"state":"active+clean","last_fresh":"2026-02-22T21:54:43.978636+0000","last_change":"2026-02-22T21:54:43.978636+0000","last_active":"2026-02-22T21:54:43.978636+0000","last_peered":"2026-02-22T21:54:43.978636+0000","last_clean":"2026-02-22T21:54:43.978636+0000","last_became_active":"2026-02-22T21:54:41.977649+0000","last_became_peered":"2026-02-22T21:54:41.977649+0000","last_unstale":"2026-02-22T21:54:43.978636+0000","last_undegraded":"2026-02-22T21:54:43.978636+0000","last_fullsized":"2026-02-22T21:54:43.978636+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T21:54:40.966951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T21:54:40.966951+0000","last_clean_scrub_stamp":"2026-02-22T21:54:40.966951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T01:28:56.435752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00078215399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":19,"reported_epoch":14,"state":"active+clean","last_fresh":"2026-02-22T21:54:43.978182+0000","last_change":"2026-02-22T21:54:43.978182+0000","last_active":"2026-02-22T21:54:43.978182+0000","last_peered":"2026-02-22T21:54:43.978182+0000","last_clean":"2026-02-22T21:54:43.978182+0000","last_became_active":"2026-02-22T21:54:41.977655+0000","last_became_peered":"2026-02-22T21:54:41.977655+0000","last_unstale":"2026-02-22T21:54:43.978182+0000","last_undegraded":"2026-02-22T21:54:43.978182+0000","last_fullsized":"2026-02-22T21:54:43.978182+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T21:54:40.966951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T21:54:40.966951+0000","last_clean_scrub_stamp":"2026-02-22T21:54:40.966951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:17:02.624184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00038385300000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":19,"reported_epoch":14,"state":"active+clean","last_fresh":"2026-02-22T21:54:43.978364+0000","last_change":"2026-02-22T21:54:43.978364+0000","last_active":"2026-02-22T21:54:43.978364+0000","last_peered":"2026-02-22T21:54:43.978364+0000","last_clean":"2026-02-22T21:54:43.978364+0000","last_became_active":"2026-02-22T21:54:41.977538+0000","last_became_peered":"2026-02-22T21:54:41.977538+0000","last_unstale":"2026-02-22T21:54:43.978364+0000","last_undegraded":"2026-02-22T21:54:43.978364+0000","last_fullsized":"2026-02-22T21:54:43.978364+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T21:54:40.966951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T21:54:40.966951+0000","last_clean_scrub_stamp":"2026-02-22T21:54:40.966951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T01:18:23.057694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057796400000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.2","version":"14'2","reported_seq":21,"reported_epoch":14,"state":"active+clean","last_fresh":"2026-02-22T21:54:43.977268+0000","last_change":"2026-02-22T21:54:43.976742+0000","last_active":"2026-02-22T21:54:43.977268+0000","last_peered":"2026-02-22T21:54:43.977268+0000","last_clean":"2026-02-22T21:54:43.977268+0000","last_became_active":"2026-02-22T21:54:41.976693+0000","last_became_peered":"2026-02-22T21:54:41.976693+0000","last_unstale":"2026-02-22T21:54:43.977268+0000","last_undegraded":"2026-02-22T21:54:43.977268+0000","last_fullsized":"2026-02-22T21:54:43.977268+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T21:54:40.966951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T21:54:40.966951+0000","last_clean_scrub_stamp":"2026-02-22T21:54:40.966951+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T06:04:27.049416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057252400000000004,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":19,"reported_epoch":14,"state":"active+clean","last_fresh":"2026-02-22T21:54:43.978409+0000","last_change":"2026-02-22T21:54:43.978409+0000","last_active":"2026-02-22T21:54:43.978409+0000","last_peered":"2026-02-22T21:54:43.978409+0000","last_clean":"2026-02-22T21:54:43.978409+0000","last_became_active":"2026-02-22T21:54:41.977454+0000","last_became_peered":"2026-02-22T21:54:41.977454+0000","last_unstale":"2026-02-22T21:54:43.978409+0000","last_undegraded":"2026-02-22T21:54:43.978409+0000","last_fullsized":"2026-02-22T21:54:43.978409+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T21:54:40.966951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T21:54:40.966951+0000","last_clean_scrub_stamp":"2026-02-22T21:54:40.966951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:01:40.151823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000616426,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":19,"reported_epoch":14,"state":"active+clean","last_fresh":"2026-02-22T21:54:43.976467+0000","last_change":"2026-02-22T21:54:43.976467+0000","last_active":"2026-02-22T21:54:43.976467+0000","last_peered":"2026-02-22T21:54:43.976467+0000","last_clean":"2026-02-22T21:54:43.976467+0000","last_became_active":"2026-02-22T21:54:41.977088+0000","last_became_peered":"2026-02-22T21:54:41.977088+0000","last_unstale":"2026-02-22T21:54:43.976467+0000","last_undegraded":"2026-02-22T21:54:43.976467+0000","last_fullsized":"2026-02-22T21:54:43.976467+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T21:54:40.966951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T21:54:40.966951+0000","last_clean_scrub_stamp":"2026-02-22T21:54:40.966951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:11:36.719854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00028617200000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.3","version":"12'1","reported_seq":20,"reported_epoch":14,"state":"active+clean","last_fresh":"2026-02-22T21:54:43.978186+0000","last_change":"2026-02-22T21:54:43.978186+0000","last_active":"2026-02-22T21:54:43.978186+0000","last_peered":"2026-02-22T21:54:43.978186+0000","last_clean":"2026-02-22T21:54:43.978186+0000","last_became_active":"2026-02-22T21:54:41.977469+0000","last_became_peered":"2026-02-22T21:54:41.977469+0000","last_unstale":"2026-02-22T21:54:43.978186+0000","last_undegraded":"2026-02-22T21:54:43.978186+0000","last_fullsized":"2026-02-22T21:54:43.978186+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T21:54:40.966951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T21:54:40.966951+0000","last_clean_scrub_stamp":"2026-02-22T21:54:40.966951+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T09:17:09.429877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039984299999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"9'52","reported_seq":87,"reported_epoch":14,"state":"active+clean","last_fresh":"2026-02-22T21:54:43.977769+0000","last_change":"2026-02-22T21:54:37.359016+0000","last_active":"2026-02-22T21:54:43.977769+0000","last_peered":"2026-02-22T21:54:43.977769+0000","last_clean":"2026-02-22T21:54:43.977769+0000","last_became_active":"2026-02-22T21:54:37.358358+0000","last_became_peered":"2026-02-22T21:54:37.358358+0000","last_unstale":"2026-02-22T21:54:43.977769+0000","last_undegraded":"2026-02-22T21:54:43.977769+0000","last_fullsized":"2026-02-22T21:54:43.977769+0000","mapping_epoch":8,"log_start":"0'0","ondisk_log_start":"0'0","created":8,"last_epoch_clean":9,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T21:54:35.955267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T21:54:35.955267+0000","last_clean_scrub_stamp":"2026-02-22T21:54:35.955267+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:52:17.396301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":56,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":2},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":56,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":52,"ondisk_log_size":52,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":1,"up_from":7,"seq":30064771077,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27668,"kb_used_data":704,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344172,"statfs":{"total":96636764160,"available":96608432128,"internally_reserved":0,"allocated":720896,"data_stored":623122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27589062},"hb_peers":[0],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":7,"seq":30064771076,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27668,"kb_used_data":696,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344172,"statfs":{"total":96636764160,"available":96608432128,"internally_reserved":0,"allocated":712704,"data_stored":619983,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27589062},"hb_peers":[1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T21:54:46.177 DEBUG:teuthology.orchestra.run.trial093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T21:54:46.328 INFO:teuthology.orchestra.run.trial093.stdout: 2026-02-22T21:54:46.328 INFO:teuthology.orchestra.run.trial093.stderr:dumped all 2026-02-22T21:54:46.337 INFO:teuthology.orchestra.run.trial093.stdout:{"pg_ready":true,"pg_map":{"version":18,"stamp":"2026-02-22T21:54:44.948939+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":56,"num_read_kb":46,"num_write":81,"num_write_kb":1108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":55,"ondisk_log_size":55,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":55336,"kb_used_data":1400,"kb_used_omap":3,"kb_used_meta":53884,"kb_avail":188688344,"statfs":{"total":193273528320,"available":193216864256,"internally_reserved":0,"allocated":1433600,"data_stored":1243105,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":3188,"internal_metadata":55178124},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"3.978694"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":19,"reported_epoch":14,"state":"active+clean","last_fresh":"2026-02-22T21:54:43.978642+0000","last_change":"2026-02-22T21:54:43.978642+0000","last_active":"2026-02-22T21:54:43.978642+0000","last_peered":"2026-02-22T21:54:43.978642+0000","last_clean":"2026-02-22T21:54:43.978642+0000","last_became_active":"2026-02-22T21:54:41.977824+0000","last_became_peered":"2026-02-22T21:54:41.977824+0000","last_unstale":"2026-02-22T21:54:43.978642+0000","last_undegraded":"2026-02-22T21:54:43.978642+0000","last_fullsized":"2026-02-22T21:54:43.978642+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T21:54:40.966951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T21:54:40.966951+0000","last_clean_scrub_stamp":"2026-02-22T21:54:40.966951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:31:00.749250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00078258400000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":19,"reported_epoch":14,"state":"active+clean","last_fresh":"2026-02-22T21:54:43.978636+0000","last_change":"2026-02-22T21:54:43.978636+0000","last_active":"2026-02-22T21:54:43.978636+0000","last_peered":"2026-02-22T21:54:43.978636+0000","last_clean":"2026-02-22T21:54:43.978636+0000","last_became_active":"2026-02-22T21:54:41.977649+0000","last_became_peered":"2026-02-22T21:54:41.977649+0000","last_unstale":"2026-02-22T21:54:43.978636+0000","last_undegraded":"2026-02-22T21:54:43.978636+0000","last_fullsized":"2026-02-22T21:54:43.978636+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T21:54:40.966951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T21:54:40.966951+0000","last_clean_scrub_stamp":"2026-02-22T21:54:40.966951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T01:28:56.435752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00078215399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":19,"reported_epoch":14,"state":"active+clean","last_fresh":"2026-02-22T21:54:43.978182+0000","last_change":"2026-02-22T21:54:43.978182+0000","last_active":"2026-02-22T21:54:43.978182+0000","last_peered":"2026-02-22T21:54:43.978182+0000","last_clean":"2026-02-22T21:54:43.978182+0000","last_became_active":"2026-02-22T21:54:41.977655+0000","last_became_peered":"2026-02-22T21:54:41.977655+0000","last_unstale":"2026-02-22T21:54:43.978182+0000","last_undegraded":"2026-02-22T21:54:43.978182+0000","last_fullsized":"2026-02-22T21:54:43.978182+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T21:54:40.966951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T21:54:40.966951+0000","last_clean_scrub_stamp":"2026-02-22T21:54:40.966951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:17:02.624184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00038385300000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":19,"reported_epoch":14,"state":"active+clean","last_fresh":"2026-02-22T21:54:43.978364+0000","last_change":"2026-02-22T21:54:43.978364+0000","last_active":"2026-02-22T21:54:43.978364+0000","last_peered":"2026-02-22T21:54:43.978364+0000","last_clean":"2026-02-22T21:54:43.978364+0000","last_became_active":"2026-02-22T21:54:41.977538+0000","last_became_peered":"2026-02-22T21:54:41.977538+0000","last_unstale":"2026-02-22T21:54:43.978364+0000","last_undegraded":"2026-02-22T21:54:43.978364+0000","last_fullsized":"2026-02-22T21:54:43.978364+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T21:54:40.966951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T21:54:40.966951+0000","last_clean_scrub_stamp":"2026-02-22T21:54:40.966951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T01:18:23.057694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057796400000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.2","version":"14'2","reported_seq":21,"reported_epoch":14,"state":"active+clean","last_fresh":"2026-02-22T21:54:43.977268+0000","last_change":"2026-02-22T21:54:43.976742+0000","last_active":"2026-02-22T21:54:43.977268+0000","last_peered":"2026-02-22T21:54:43.977268+0000","last_clean":"2026-02-22T21:54:43.977268+0000","last_became_active":"2026-02-22T21:54:41.976693+0000","last_became_peered":"2026-02-22T21:54:41.976693+0000","last_unstale":"2026-02-22T21:54:43.977268+0000","last_undegraded":"2026-02-22T21:54:43.977268+0000","last_fullsized":"2026-02-22T21:54:43.977268+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T21:54:40.966951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T21:54:40.966951+0000","last_clean_scrub_stamp":"2026-02-22T21:54:40.966951+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T06:04:27.049416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057252400000000004,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":19,"reported_epoch":14,"state":"active+clean","last_fresh":"2026-02-22T21:54:43.978409+0000","last_change":"2026-02-22T21:54:43.978409+0000","last_active":"2026-02-22T21:54:43.978409+0000","last_peered":"2026-02-22T21:54:43.978409+0000","last_clean":"2026-02-22T21:54:43.978409+0000","last_became_active":"2026-02-22T21:54:41.977454+0000","last_became_peered":"2026-02-22T21:54:41.977454+0000","last_unstale":"2026-02-22T21:54:43.978409+0000","last_undegraded":"2026-02-22T21:54:43.978409+0000","last_fullsized":"2026-02-22T21:54:43.978409+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T21:54:40.966951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T21:54:40.966951+0000","last_clean_scrub_stamp":"2026-02-22T21:54:40.966951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:01:40.151823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000616426,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":19,"reported_epoch":14,"state":"active+clean","last_fresh":"2026-02-22T21:54:43.976467+0000","last_change":"2026-02-22T21:54:43.976467+0000","last_active":"2026-02-22T21:54:43.976467+0000","last_peered":"2026-02-22T21:54:43.976467+0000","last_clean":"2026-02-22T21:54:43.976467+0000","last_became_active":"2026-02-22T21:54:41.977088+0000","last_became_peered":"2026-02-22T21:54:41.977088+0000","last_unstale":"2026-02-22T21:54:43.976467+0000","last_undegraded":"2026-02-22T21:54:43.976467+0000","last_fullsized":"2026-02-22T21:54:43.976467+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T21:54:40.966951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T21:54:40.966951+0000","last_clean_scrub_stamp":"2026-02-22T21:54:40.966951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:11:36.719854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00028617200000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.3","version":"12'1","reported_seq":20,"reported_epoch":14,"state":"active+clean","last_fresh":"2026-02-22T21:54:43.978186+0000","last_change":"2026-02-22T21:54:43.978186+0000","last_active":"2026-02-22T21:54:43.978186+0000","last_peered":"2026-02-22T21:54:43.978186+0000","last_clean":"2026-02-22T21:54:43.978186+0000","last_became_active":"2026-02-22T21:54:41.977469+0000","last_became_peered":"2026-02-22T21:54:41.977469+0000","last_unstale":"2026-02-22T21:54:43.978186+0000","last_undegraded":"2026-02-22T21:54:43.978186+0000","last_fullsized":"2026-02-22T21:54:43.978186+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T21:54:40.966951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T21:54:40.966951+0000","last_clean_scrub_stamp":"2026-02-22T21:54:40.966951+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T09:17:09.429877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039984299999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"9'52","reported_seq":87,"reported_epoch":14,"state":"active+clean","last_fresh":"2026-02-22T21:54:43.977769+0000","last_change":"2026-02-22T21:54:37.359016+0000","last_active":"2026-02-22T21:54:43.977769+0000","last_peered":"2026-02-22T21:54:43.977769+0000","last_clean":"2026-02-22T21:54:43.977769+0000","last_became_active":"2026-02-22T21:54:37.358358+0000","last_became_peered":"2026-02-22T21:54:37.358358+0000","last_unstale":"2026-02-22T21:54:43.977769+0000","last_undegraded":"2026-02-22T21:54:43.977769+0000","last_fullsized":"2026-02-22T21:54:43.977769+0000","mapping_epoch":8,"log_start":"0'0","ondisk_log_start":"0'0","created":8,"last_epoch_clean":9,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T21:54:35.955267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T21:54:35.955267+0000","last_clean_scrub_stamp":"2026-02-22T21:54:35.955267+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:52:17.396301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":56,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":2},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":56,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":52,"ondisk_log_size":52,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":1,"up_from":7,"seq":30064771077,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27668,"kb_used_data":704,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344172,"statfs":{"total":96636764160,"available":96608432128,"internally_reserved":0,"allocated":720896,"data_stored":623122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27589062},"hb_peers":[0],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":7,"seq":30064771076,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27668,"kb_used_data":696,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344172,"statfs":{"total":96636764160,"available":96608432128,"internally_reserved":0,"allocated":712704,"data_stored":619983,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27589062},"hb_peers":[1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T21:54:46.339 INFO:tasks.ceph.ceph_manager.ceph:clean! 2026-02-22T21:54:46.339 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2026-02-22T21:54:46.339 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2026-02-22T21:54:46.339 DEBUG:teuthology.orchestra.run.trial093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-22T21:54:46.499 INFO:teuthology.orchestra.run.trial093.stdout: 2026-02-22T21:54:46.499 INFO:teuthology.orchestra.run.trial093.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2026-02-22T21:54:46.510 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2026-02-22T21:54:46.510 INFO:teuthology.run_tasks:Running task workunit... 2026-02-22T21:54:46.547 INFO:tasks.workunit:Pulling workunits from ref f90f4eaebb28b619cf76e80cbbe997656da69579 2026-02-22T21:54:46.548 INFO:tasks.workunit:Making a separate scratch dir for every client... 2026-02-22T21:54:46.548 INFO:tasks.workunit:timeout=3h 2026-02-22T21:54:46.548 INFO:tasks.workunit:cleanup=True 2026-02-22T21:54:46.549 DEBUG:teuthology.orchestra.run.trial093:> stat -- /home/ubuntu/cephtest/mnt.0 2026-02-22T21:54:46.553 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-22T21:54:46.553 INFO:teuthology.orchestra.run.trial093.stderr:stat: cannot statx '/home/ubuntu/cephtest/mnt.0': No such file or directory 2026-02-22T21:54:46.553 DEBUG:teuthology.orchestra.run.trial093:> mkdir -- /home/ubuntu/cephtest/mnt.0 2026-02-22T21:54:46.599 INFO:tasks.workunit:Created dir /home/ubuntu/cephtest/mnt.0 2026-02-22T21:54:46.600 DEBUG:teuthology.orchestra.run.trial093:> cd -- /home/ubuntu/cephtest/mnt.0 && mkdir -- client.0 2026-02-22T21:54:46.648 DEBUG:teuthology.orchestra.run.trial093:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://github.com/ceph/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout f90f4eaebb28b619cf76e80cbbe997656da69579 2026-02-22T21:54:46.698 INFO:tasks.workunit.client.0.trial093.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2026-02-22T21:55:09.179 INFO:tasks.workunit.client.0.trial093.stderr:Note: switching to 'f90f4eaebb28b619cf76e80cbbe997656da69579'. 2026-02-22T21:55:09.180 INFO:tasks.workunit.client.0.trial093.stderr: 2026-02-22T21:55:09.180 INFO:tasks.workunit.client.0.trial093.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2026-02-22T21:55:09.180 INFO:tasks.workunit.client.0.trial093.stderr:changes and commit them, and you can discard any commits you make in this 2026-02-22T21:55:09.180 INFO:tasks.workunit.client.0.trial093.stderr:state without impacting any branches by switching back to a branch. 2026-02-22T21:55:09.180 INFO:tasks.workunit.client.0.trial093.stderr: 2026-02-22T21:55:09.180 INFO:tasks.workunit.client.0.trial093.stderr:If you want to create a new branch to retain commits you create, you may 2026-02-22T21:55:09.180 INFO:tasks.workunit.client.0.trial093.stderr:do so (now or later) by using -c with the switch command. Example: 2026-02-22T21:55:09.180 INFO:tasks.workunit.client.0.trial093.stderr: 2026-02-22T21:55:09.180 INFO:tasks.workunit.client.0.trial093.stderr: git switch -c 2026-02-22T21:55:09.180 INFO:tasks.workunit.client.0.trial093.stderr: 2026-02-22T21:55:09.180 INFO:tasks.workunit.client.0.trial093.stderr:Or undo this operation with: 2026-02-22T21:55:09.180 INFO:tasks.workunit.client.0.trial093.stderr: 2026-02-22T21:55:09.180 INFO:tasks.workunit.client.0.trial093.stderr: git switch - 2026-02-22T21:55:09.180 INFO:tasks.workunit.client.0.trial093.stderr: 2026-02-22T21:55:09.180 INFO:tasks.workunit.client.0.trial093.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2026-02-22T21:55:09.180 INFO:tasks.workunit.client.0.trial093.stderr: 2026-02-22T21:55:09.180 INFO:tasks.workunit.client.0.trial093.stderr:HEAD is now at f90f4eaebb2 Merge branch 'wip-74712-squid' of https://github.com/idryomov/ceph into wip-yuri6-testing-2026-02-22-1629-squid 2026-02-22T21:55:09.189 DEBUG:teuthology.orchestra.run.trial093:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2026-02-22T21:55:09.235 INFO:tasks.workunit.client.0.trial093.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2026-02-22T21:55:09.237 INFO:tasks.workunit.client.0.trial093.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2026-02-22T21:55:09.237 INFO:tasks.workunit.client.0.trial093.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2026-02-22T21:55:09.331 INFO:tasks.workunit.client.0.trial093.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2026-02-22T21:55:09.363 INFO:tasks.workunit.client.0.trial093.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2026-02-22T21:55:09.390 INFO:tasks.workunit.client.0.trial093.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2026-02-22T21:55:09.392 INFO:tasks.workunit.client.0.trial093.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2026-02-22T21:55:09.392 INFO:tasks.workunit.client.0.trial093.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2026-02-22T21:55:09.414 INFO:tasks.workunit.client.0.trial093.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2026-02-22T21:55:09.417 DEBUG:teuthology.orchestra.run.trial093:> set -ex 2026-02-22T21:55:09.418 DEBUG:teuthology.orchestra.run.trial093:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2026-02-22T21:55:09.464 INFO:tasks.workunit:Running workunits matching rbd/test_admin_socket.sh on client.0... 2026-02-22T21:55:09.467 INFO:tasks.workunit:Running workunit rbd/test_admin_socket.sh... 2026-02-22T21:55:09.468 DEBUG:teuthology.orchestra.run.trial093:workunit test rbd/test_admin_socket.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=f90f4eaebb28b619cf76e80cbbe997656da69579 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rbd/test_admin_socket.sh 2026-02-22T21:55:09.514 INFO:tasks.workunit.client.0.trial093.stderr:+ TMPDIR=/tmp/rbd_test_admin_socket17344 2026-02-22T21:55:09.515 INFO:tasks.workunit.client.0.trial093.stderr:+ mkdir /tmp/rbd_test_admin_socket17344 2026-02-22T21:55:09.515 INFO:tasks.workunit.client.0.trial093.stderr:+ trap 'rm -fr /tmp/rbd_test_admin_socket17344' 0 2026-02-22T21:55:09.516 INFO:tasks.workunit.client.0.trial093.stderr:++ dirname /home/ubuntu/cephtest/clone.client.0/qa/workunits/rbd/test_admin_socket.sh 2026-02-22T21:55:09.517 INFO:tasks.workunit.client.0.trial093.stderr:+ . /home/ubuntu/cephtest/clone.client.0/qa/workunits/rbd/../../standalone/ceph-helpers.sh 2026-02-22T21:55:09.517 INFO:tasks.workunit.client.0.trial093.stderr:++ TIMEOUT=300 2026-02-22T21:55:09.517 INFO:tasks.workunit.client.0.trial093.stderr:++ WAIT_FOR_CLEAN_TIMEOUT=90 2026-02-22T21:55:09.517 INFO:tasks.workunit.client.0.trial093.stderr:++ MAX_TIMEOUT=15 2026-02-22T21:55:09.517 INFO:tasks.workunit.client.0.trial093.stderr:++ PG_NUM=4 2026-02-22T21:55:09.517 INFO:tasks.workunit.client.0.trial093.stderr:++ TMPDIR=/tmp/rbd_test_admin_socket17344 2026-02-22T21:55:09.517 INFO:tasks.workunit.client.0.trial093.stderr:++ CEPH_BUILD_VIRTUALENV=/tmp/rbd_test_admin_socket17344 2026-02-22T21:55:09.518 INFO:tasks.workunit.client.0.trial093.stderr:++ TESTDIR=/home/ubuntu/cephtest 2026-02-22T21:55:09.518 INFO:tasks.workunit.client.0.trial093.stderr:++ type xmlstarlet 2026-02-22T21:55:09.518 INFO:tasks.workunit.client.0.trial093.stderr:++ XMLSTARLET=xmlstarlet 2026-02-22T21:55:09.518 INFO:tasks.workunit.client.0.trial093.stderr:+++ uname 2026-02-22T21:55:09.519 INFO:tasks.workunit.client.0.trial093.stderr:++ '[' Linux = FreeBSD ']' 2026-02-22T21:55:09.519 INFO:tasks.workunit.client.0.trial093.stderr:++ SED=sed 2026-02-22T21:55:09.519 INFO:tasks.workunit.client.0.trial093.stderr:++ AWK=awk 2026-02-22T21:55:09.519 INFO:tasks.workunit.client.0.trial093.stderr:+++ stty -a 2026-02-22T21:55:09.519 INFO:tasks.workunit.client.0.trial093.stderr:+++ head -1 2026-02-22T21:55:09.519 INFO:tasks.workunit.client.0.trial093.stderr:+++ sed -e 's/.*columns \([0-9]*\).*/\1/' 2026-02-22T21:55:09.520 INFO:tasks.workunit.client.0.trial093.stderr:stty: 'standard input': Inappropriate ioctl for device 2026-02-22T21:55:09.520 INFO:tasks.workunit.client.0.trial093.stderr:++ termwidth= 2026-02-22T21:55:09.520 INFO:tasks.workunit.client.0.trial093.stderr:++ '[' -n '' -a '' '!=' 0 ']' 2026-02-22T21:55:09.520 INFO:tasks.workunit.client.0.trial093.stderr:++ DIFFCOLOPTS='-y ' 2026-02-22T21:55:09.520 INFO:tasks.workunit.client.0.trial093.stderr:++ KERNCORE=kernel.core_pattern 2026-02-22T21:55:09.521 INFO:tasks.workunit.client.0.trial093.stderr:++ EXTRA_OPTS= 2026-02-22T21:55:09.521 INFO:tasks.workunit.client.0.trial093.stderr:++ test '' = TESTS 2026-02-22T21:55:09.522 INFO:tasks.workunit.client.0.trial093.stderr:+ pool=rbd 2026-02-22T21:55:09.522 INFO:tasks.workunit.client.0.trial093.stderr:+ image=testimg17344 2026-02-22T21:55:09.522 INFO:tasks.workunit.client.0.trial093.stderr:++ rbd_watch_asok testimg17344 2026-02-22T21:55:09.523 INFO:tasks.workunit.client.0.trial093.stderr:++ echo /tmp/rbd_test_admin_socket17344/rbd_watch_testimg17344.asok 2026-02-22T21:55:09.523 INFO:tasks.workunit.client.0.trial093.stderr:+ ceph_admin='ceph --admin-daemon /tmp/rbd_test_admin_socket17344/rbd_watch_testimg17344.asok' 2026-02-22T21:55:09.523 INFO:tasks.workunit.client.0.trial093.stderr:+ rbd create --size 128 rbd/testimg17344 2026-02-22T21:55:09.553 INFO:tasks.workunit.client.0.trial093.stderr:+ rbd_cache_flush='rbd cache flush rbd/testimg17344' 2026-02-22T21:55:09.553 INFO:tasks.workunit.client.0.trial093.stderr:+ rbd_cache_invalidate='rbd cache invalidate rbd/testimg17344' 2026-02-22T21:55:09.553 INFO:tasks.workunit.client.0.trial093.stderr:+ rbd_watch_start testimg17344 2026-02-22T21:55:09.553 INFO:tasks.workunit.client.0.trial093.stderr:+ local image=testimg17344 2026-02-22T21:55:09.553 INFO:tasks.workunit.client.0.trial093.stderr:++ rbd_watch_asok testimg17344 2026-02-22T21:55:09.553 INFO:tasks.workunit.client.0.trial093.stderr:++ echo /tmp/rbd_test_admin_socket17344/rbd_watch_testimg17344.asok 2026-02-22T21:55:09.553 INFO:tasks.workunit.client.0.trial093.stderr:+ local asok=/tmp/rbd_test_admin_socket17344/rbd_watch_testimg17344.asok 2026-02-22T21:55:09.554 INFO:tasks.workunit.client.0.trial093.stderr:++ rbd_watch_fifo testimg17344 2026-02-22T21:55:09.554 INFO:tasks.workunit.client.0.trial093.stderr:++ echo /tmp/rbd_test_admin_socket17344/rbd_watch_testimg17344.fifo 2026-02-22T21:55:09.554 INFO:tasks.workunit.client.0.trial093.stderr:+ mkfifo /tmp/rbd_test_admin_socket17344/rbd_watch_testimg17344.fifo 2026-02-22T21:55:09.555 INFO:tasks.workunit.client.0.trial093.stderr:+ local pid 2026-02-22T21:55:09.556 INFO:tasks.workunit.client.0.trial093.stderr:++ seq 10 2026-02-22T21:55:09.557 INFO:tasks.workunit.client.0.trial093.stderr:+ rbd --admin-socket /tmp/rbd_test_admin_socket17344/rbd_watch_testimg17344.asok watch testimg17344 2026-02-22T21:55:09.557 INFO:tasks.workunit.client.0.trial093.stderr:++ rbd_watch_fifo testimg17344 2026-02-22T21:55:09.557 INFO:tasks.workunit.client.0.trial093.stderr:++ echo /tmp/rbd_test_admin_socket17344/rbd_watch_testimg17344.fifo 2026-02-22T21:55:09.557 INFO:tasks.workunit.client.0.trial093.stderr:++ rbd_watch_out_file testimg17344 2026-02-22T21:55:09.557 INFO:tasks.workunit.client.0.trial093.stderr:++ echo /tmp/rbd_test_admin_socket17344/rbd_watch_testimg17344.out 2026-02-22T21:55:09.557 INFO:tasks.workunit.client.0.trial093.stderr:+ cat /tmp/rbd_test_admin_socket17344/rbd_watch_testimg17344.fifo 2026-02-22T21:55:09.557 INFO:tasks.workunit.client.0.trial093.stderr:+ for i in `seq 10` 2026-02-22T21:55:09.558 INFO:tasks.workunit.client.0.trial093.stderr:++ ps auxww 2026-02-22T21:55:09.558 INFO:tasks.workunit.client.0.trial093.stderr:++ awk '/[r]bd --admin.* watch testimg17344/ {print $2}' 2026-02-22T21:55:09.571 INFO:tasks.workunit.client.0.trial093.stderr:+ pid=17376 2026-02-22T21:55:09.572 INFO:tasks.workunit.client.0.trial093.stderr:+ test -n 17376 2026-02-22T21:55:09.572 INFO:tasks.workunit.client.0.trial093.stderr:+ break 2026-02-22T21:55:09.572 INFO:tasks.workunit.client.0.trial093.stderr:+ test -n 17376 2026-02-22T21:55:09.572 INFO:tasks.workunit.client.0.trial093.stderr:+ echo 17376 2026-02-22T21:55:09.572 INFO:tasks.workunit.client.0.trial093.stderr:++ rbd_watch_pid_file testimg17344 2026-02-22T21:55:09.572 INFO:tasks.workunit.client.0.trial093.stderr:++ echo /tmp/rbd_test_admin_socket17344/rbd_watch_testimg17344.pid 2026-02-22T21:55:09.573 INFO:tasks.workunit.client.0.trial093.stderr:+ test -n /tmp/rbd_test_admin_socket17344/rbd_watch_testimg17344.asok 2026-02-22T21:55:09.573 INFO:tasks.workunit.client.0.trial093.stderr:++ seq 10 2026-02-22T21:55:09.573 INFO:tasks.workunit.client.0.trial093.stderr:+ for i in `seq 10` 2026-02-22T21:55:09.573 INFO:tasks.workunit.client.0.trial093.stderr:+ test -S /tmp/rbd_test_admin_socket17344/rbd_watch_testimg17344.asok 2026-02-22T21:55:09.573 INFO:tasks.workunit.client.0.trial093.stderr:+ sleep 0.1 2026-02-22T21:55:09.672 INFO:tasks.workunit.client.0.trial093.stderr:+ for i in `seq 10` 2026-02-22T21:55:09.672 INFO:tasks.workunit.client.0.trial093.stderr:+ test -S /tmp/rbd_test_admin_socket17344/rbd_watch_testimg17344.asok 2026-02-22T21:55:09.673 INFO:tasks.workunit.client.0.trial093.stderr:+ break 2026-02-22T21:55:09.673 INFO:tasks.workunit.client.0.trial093.stderr:+ test -S /tmp/rbd_test_admin_socket17344/rbd_watch_testimg17344.asok 2026-02-22T21:55:09.673 INFO:tasks.workunit.client.0.trial093.stderr:+ ceph --admin-daemon /tmp/rbd_test_admin_socket17344/rbd_watch_testimg17344.asok config set debug_rbd 20 2026-02-22T21:55:09.722 INFO:tasks.workunit.client.0.trial093.stdout:{ 2026-02-22T21:55:09.722 INFO:tasks.workunit.client.0.trial093.stdout: "success": "" 2026-02-22T21:55:09.722 INFO:tasks.workunit.client.0.trial093.stdout:} 2026-02-22T21:55:09.728 INFO:tasks.workunit.client.0.trial093.stderr:+ rbd status testimg17344 2026-02-22T21:55:09.728 INFO:tasks.workunit.client.0.trial093.stderr:+ expect_false grep 'Watchers: none' 2026-02-22T21:55:09.728 INFO:tasks.workunit.client.0.trial093.stderr:+ set -x 2026-02-22T21:55:09.728 INFO:tasks.workunit.client.0.trial093.stderr:+ grep 'Watchers: none' 2026-02-22T21:55:09.759 INFO:tasks.workunit.client.0.trial093.stderr:+ return 0 2026-02-22T21:55:09.760 INFO:tasks.workunit.client.0.trial093.stderr:+ ceph --admin-daemon /tmp/rbd_test_admin_socket17344/rbd_watch_testimg17344.asok help 2026-02-22T21:55:09.760 INFO:tasks.workunit.client.0.trial093.stderr:+ fgrep 'rbd cache flush rbd/testimg17344' 2026-02-22T21:55:09.815 INFO:tasks.workunit.client.0.trial093.stdout: "rbd cache flush rbd/testimg17344": "flush rbd image rbd/testimg17344 cache", 2026-02-22T21:55:09.816 INFO:tasks.workunit.client.0.trial093.stderr:+ ceph --admin-daemon /tmp/rbd_test_admin_socket17344/rbd_watch_testimg17344.asok help 2026-02-22T21:55:09.816 INFO:tasks.workunit.client.0.trial093.stderr:+ fgrep 'rbd cache invalidate rbd/testimg17344' 2026-02-22T21:55:09.870 INFO:tasks.workunit.client.0.trial093.stdout: "rbd cache invalidate rbd/testimg17344": "invalidate rbd image rbd/testimg17344 cache", 2026-02-22T21:55:09.871 INFO:tasks.workunit.client.0.trial093.stderr:+ rbd_watch_end testimg17344 2026-02-22T21:55:09.871 INFO:tasks.workunit.client.0.trial093.stderr:+ local image=testimg17344 2026-02-22T21:55:09.871 INFO:tasks.workunit.client.0.trial093.stderr:+ local regexp= 2026-02-22T21:55:09.871 INFO:tasks.workunit.client.0.trial093.stderr:+ echo 2026-02-22T21:55:09.872 INFO:tasks.workunit.client.0.trial093.stderr:++ rbd_watch_fifo testimg17344 2026-02-22T21:55:09.872 INFO:tasks.workunit.client.0.trial093.stderr:++ echo /tmp/rbd_test_admin_socket17344/rbd_watch_testimg17344.fifo 2026-02-22T21:55:09.872 INFO:tasks.workunit.client.0.trial093.stderr:+++ rbd_watch_pid_file testimg17344 2026-02-22T21:55:09.872 INFO:tasks.workunit.client.0.trial093.stderr:+++ echo /tmp/rbd_test_admin_socket17344/rbd_watch_testimg17344.pid 2026-02-22T21:55:09.873 INFO:tasks.workunit.client.0.trial093.stderr:++ cat /tmp/rbd_test_admin_socket17344/rbd_watch_testimg17344.pid 2026-02-22T21:55:09.873 INFO:tasks.workunit.client.0.trial093.stderr:+ kill 17376 2026-02-22T21:55:09.873 INFO:tasks.workunit.client.0.trial093.stderr:++ rbd_watch_out_file testimg17344 2026-02-22T21:55:09.874 INFO:tasks.workunit.client.0.trial093.stderr:++ echo /tmp/rbd_test_admin_socket17344/rbd_watch_testimg17344.out 2026-02-22T21:55:09.874 INFO:tasks.workunit.client.0.trial093.stderr:+ cat /tmp/rbd_test_admin_socket17344/rbd_watch_testimg17344.out 2026-02-22T21:55:09.874 INFO:tasks.workunit.client.0.trial093.stdout:press enter to exit... 2026-02-22T21:55:09.874 INFO:tasks.workunit.client.0.trial093.stdout:2026-02-22T21:55:09.870+0000 7f1c6bb18080 20 librbd::ImageState: 0x563909df7c20 close 2026-02-22T21:55:09.875 INFO:tasks.workunit.client.0.trial093.stdout:2026-02-22T21:55:09.870+0000 7f1c6bb18080 10 librbd::ImageState: 0x563909df7c20 0x563909df7c20 send_close_unlock 2026-02-22T21:55:09.875 INFO:tasks.workunit.client.0.trial093.stdout:2026-02-22T21:55:09.870+0000 7f1c6bb18080 10 librbd::ConfigWatcher: shut_down: 2026-02-22T21:55:09.875 INFO:tasks.workunit.client.0.trial093.stdout:2026-02-22T21:55:09.870+0000 7f1c6bb18080 10 librbd::image::CloseRequest: 0x563909e6cd90 send_block_image_watcher 2026-02-22T21:55:09.875 INFO:tasks.workunit.client.0.trial093.stdout:2026-02-22T21:55:09.870+0000 7f1c6bb18080 10 librbd::ImageWatcher: 0x7f1c4c007430 block_notifies 2026-02-22T21:55:09.875 INFO:tasks.workunit.client.0.trial093.stdout:2026-02-22T21:55:09.870+0000 7f1c6bb18080 5 librbd::Watcher: 0x7f1c4c007430 block_notifies: blocked_count=1 2026-02-22T21:55:09.875 INFO:tasks.workunit.client.0.trial093.stdout:2026-02-22T21:55:09.870+0000 7f1c6bb18080 10 librbd::image::CloseRequest: 0x563909e6cd90 handle_block_image_watcher: r=0 2026-02-22T21:55:09.875 INFO:tasks.workunit.client.0.trial093.stdout:2026-02-22T21:55:09.870+0000 7f1c6bb18080 10 librbd::image::CloseRequest: 0x563909e6cd90 send_shut_down_update_watchers 2026-02-22T21:55:09.875 INFO:tasks.workunit.client.0.trial093.stdout:2026-02-22T21:55:09.870+0000 7f1c6bb18080 20 librbd::ImageState: 0x563909df7c20 shut_down_update_watchers 2026-02-22T21:55:09.875 INFO:tasks.workunit.client.0.trial093.stdout:2026-02-22T21:55:09.870+0000 7f1c6bb18080 20 librbd::ImageState: 0x563909cc8010 ImageUpdateWatchers::shut_down 2026-02-22T21:55:09.875 INFO:tasks.workunit.client.0.trial093.stdout:2026-02-22T21:55:09.870+0000 7f1c6bb18080 20 librbd::ImageState: 0x563909cc8010 ImageUpdateWatchers::shut_down: completing shut down 2026-02-22T21:55:09.875 INFO:tasks.workunit.client.0.trial093.stdout:2026-02-22T21:55:09.870+0000 7f1c68dfa640 10 librbd::image::CloseRequest: 0x563909e6cd90 handle_shut_down_update_watchers: r=0 2026-02-22T21:55:09.876 INFO:tasks.workunit.client.0.trial093.stdout:2026-02-22T21:55:09.870+0000 7f1c68dfa640 10 librbd::image::CloseRequest: 0x563909e6cd90 send_flush 2026-02-22T21:55:09.876 INFO:tasks.workunit.client.0.trial093.stdout:2026-02-22T21:55:09.870+0000 7f1c68dfa640 20 librbd::io::AsyncOperation: 0x7f1c4c014020 start_op 2026-02-22T21:55:09.876 INFO:tasks.workunit.client.0.trial093.stdout:2026-02-22T21:55:09.870+0000 7f1c68dfa640 20 librbd::io::Dispatcher: 0x563909df8280 send: dispatch_spec=0x7f1c4c0175a0 2026-02-22T21:55:09.876 INFO:tasks.workunit.client.0.trial093.stdout:2026-02-22T21:55:09.870+0000 7f1c68dfa640 20 librbd::io::QueueImageDispatch: 0x563909e6bb20 flush: tid=1 2026-02-22T21:55:09.876 INFO:tasks.workunit.client.0.trial093.stdout:2026-02-22T21:55:09.870+0000 7f1c68dfa640 20 librbd::io::Dispatcher: 0x563909df8280 send: dispatch_spec=0x7f1c4c0175a0 2026-02-22T21:55:09.876 INFO:tasks.workunit.client.0.trial093.stdout:2026-02-22T21:55:09.870+0000 7f1c68dfa640 20 librbd::io::QosImageDispatch: 0x563909e6bc50 flush: tid=1 2026-02-22T21:55:09.876 INFO:tasks.workunit.client.0.trial093.stdout:2026-02-22T21:55:09.870+0000 7f1c68dfa640 20 librbd::io::Dispatcher: 0x563909df8280 send: dispatch_spec=0x7f1c4c0175a0 2026-02-22T21:55:09.876 INFO:tasks.workunit.client.0.trial093.stdout:2026-02-22T21:55:09.870+0000 7f1c68dfa640 20 librbd::io::RefreshImageDispatch: 0x563909e6ba60 flush: tid=1 2026-02-22T21:55:09.876 INFO:tasks.workunit.client.0.trial093.stdout:2026-02-22T21:55:09.870+0000 7f1c68dfa640 20 librbd::io::WriteBlockImageDispatch: 0x563909be04b0 flush: tid=1 2026-02-22T21:55:09.876 INFO:tasks.workunit.client.0.trial093.stdout:2026-02-22T21:55:09.870+0000 7f1c68dfa640 20 librbd::io::ImageDispatch: 0x563909e6bca0 flush: 2026-02-22T21:55:09.876 INFO:tasks.workunit.client.0.trial093.stdout:2026-02-22T21:55:09.870+0000 7f1c68dfa640 20 librbd::io::ImageRequest: 0x7f1c68df4960 send: aio_flush: ictx=0x563909e1ba50, completion=0x7f1c4c013f10 2026-02-22T21:55:09.876 INFO:tasks.workunit.client.0.trial093.stdout:2026-02-22T21:55:09.870+0000 7f1c68dfa640 20 librbd::io::AioCompletion: 0x7f1c4c013f10 set_request_count: pending=1 2026-02-22T21:55:09.877 INFO:tasks.workunit.client.0.trial093.stdout:2026-02-22T21:55:09.870+0000 7f1c68dfa640 20 librbd::io::Dispatcher: 0x563909df01f0 send: dispatch_spec=0x7f1c4c017690 2026-02-22T21:55:09.877 INFO:tasks.workunit.client.0.trial093.stdout:2026-02-22T21:55:09.870+0000 7f1c68dfa640 20 librbd::cache::WriteAroundObjectDispatch: 0x7f1c40001f00 flush: 2026-02-22T21:55:09.877 INFO:tasks.workunit.client.0.trial093.stdout:2026-02-22T21:55:09.870+0000 7f1c68dfa640 20 librbd::io::AioCompletion: 0x7f1c4c013f10 complete_request: cb=1, pending=0 2026-02-22T21:55:09.877 INFO:tasks.workunit.client.0.trial093.stdout:2026-02-22T21:55:09.870+0000 7f1c68dfa640 20 librbd::io::AioCompletion: 0x7f1c4c013f10 finalize: r=0 2026-02-22T21:55:09.877 INFO:tasks.workunit.client.0.trial093.stdout:2026-02-22T21:55:09.870+0000 7f1c68dfa640 10 librbd::image::CloseRequest: 0x563909e6cd90 handle_flush: r=0 2026-02-22T21:55:09.877 INFO:tasks.workunit.client.0.trial093.stdout:2026-02-22T21:55:09.870+0000 7f1c68dfa640 10 librbd::image::CloseRequest: 0x563909e6cd90 send_unregister_image_watcher 2026-02-22T21:55:09.877 INFO:tasks.workunit.client.0.trial093.stdout:2026-02-22T21:55:09.870+0000 7f1c68dfa640 10 librbd::ImageWatcher: 0x7f1c4c007430 unregistering image watcher 2026-02-22T21:55:09.877 INFO:tasks.workunit.client.0.trial093.stdout:2026-02-22T21:55:09.870+0000 7f1c68dfa640 10 librbd::Watcher: 0x7f1c4c007430 unregister_watch: 2026-02-22T21:55:09.877 INFO:tasks.workunit.client.0.trial093.stdout:2026-02-22T21:55:09.870+0000 7f1c68dfa640 20 librbd::io::AsyncOperation: 0x7f1c4c014020 finish_op 2026-02-22T21:55:09.877 INFO:tasks.workunit.client.0.trial093.stdout:2026-02-22T21:55:09.870+0000 7f1c46ffd640 10 librbd::image::CloseRequest: 0x563909e6cd90 handle_unregister_image_watcher: r=0 2026-02-22T21:55:09.877 INFO:tasks.workunit.client.0.trial093.stdout:2026-02-22T21:55:09.870+0000 7f1c46ffd640 10 librbd::image::CloseRequest: 0x563909e6cd90 send_flush_readahead 2026-02-22T21:55:09.877 INFO:tasks.workunit.client.0.trial093.stdout:2026-02-22T21:55:09.870+0000 7f1c5bfff640 10 librbd::image::CloseRequest: 0x563909e6cd90 handle_flush_readahead: r=0 2026-02-22T21:55:09.878 INFO:tasks.workunit.client.0.trial093.stdout:2026-02-22T21:55:09.870+0000 7f1c5bfff640 10 librbd::image::CloseRequest: 0x563909e6cd90 send_shut_down_image_dispatcher 2026-02-22T21:55:09.878 INFO:tasks.workunit.client.0.trial093.stdout:2026-02-22T21:55:09.870+0000 7f1c5bfff640 20 librbd::io::AsyncOperation: 0x7f1c38001d80 start_op 2026-02-22T21:55:09.878 INFO:tasks.workunit.client.0.trial093.stdout:2026-02-22T21:55:09.870+0000 7f1c5bfff640 5 librbd::io::Dispatcher: 0x563909df8280 shut_down: 2026-02-22T21:55:09.878 INFO:tasks.workunit.client.0.trial093.stdout:2026-02-22T21:55:09.870+0000 7f1c5bfff640 20 librbd::io::FlushTracker: 0x563909bf96c0 shut_down: 2026-02-22T21:55:09.878 INFO:tasks.workunit.client.0.trial093.stdout:2026-02-22T21:55:09.870+0000 7f1c5bfff640 20 librbd::io::FlushTracker: 0x563909c16ee0 shut_down: 2026-02-22T21:55:09.878 INFO:tasks.workunit.client.0.trial093.stdout:2026-02-22T21:55:09.870+0000 7f1c5bfff640 20 librbd::io::AsyncOperation: 0x7f1c38001d80 finish_op 2026-02-22T21:55:09.878 INFO:tasks.workunit.client.0.trial093.stdout:2026-02-22T21:55:09.870+0000 7f1c5bfff640 10 librbd::image::CloseRequest: 0x563909e6cd90 handle_shut_down_image_dispatcher: r=0 2026-02-22T21:55:09.878 INFO:tasks.workunit.client.0.trial093.stdout:2026-02-22T21:55:09.870+0000 7f1c5bfff640 10 librbd::image::CloseRequest: 0x563909e6cd90 send_shut_down_object_dispatcher 2026-02-22T21:55:09.878 INFO:tasks.workunit.client.0.trial093.stdout:2026-02-22T21:55:09.870+0000 7f1c5bfff640 5 librbd::io::Dispatcher: 0x563909df01f0 shut_down: 2026-02-22T21:55:09.878 INFO:tasks.workunit.client.0.trial093.stdout:2026-02-22T21:55:09.870+0000 7f1c5bfff640 5 librbd::io::ObjectDispatch: 0x563909e6b820 shut_down: 2026-02-22T21:55:09.878 INFO:tasks.workunit.client.0.trial093.stdout:2026-02-22T21:55:09.870+0000 7f1c5bfff640 5 librbd::cache::WriteAroundObjectDispatch: 0x7f1c40001f00 shut_down: 2026-02-22T21:55:09.878 INFO:tasks.workunit.client.0.trial093.stdout:2026-02-22T21:55:09.870+0000 7f1c5bfff640 10 librbd::image::CloseRequest: 0x563909e6cd90 handle_shut_down_object_dispatcher: r=0 2026-02-22T21:55:09.879 INFO:tasks.workunit.client.0.trial093.stdout:2026-02-22T21:55:09.870+0000 7f1c5bfff640 10 librbd::image::CloseRequest: 0x563909e6cd90 send_flush_op_work_queue 2026-02-22T21:55:09.879 INFO:tasks.workunit.client.0.trial093.stdout:2026-02-22T21:55:09.870+0000 7f1c5bfff640 10 librbd::image::CloseRequest: 0x563909e6cd90 handle_flush_op_work_queue: r=0 2026-02-22T21:55:09.879 INFO:tasks.workunit.client.0.trial093.stdout:2026-02-22T21:55:09.870+0000 7f1c5bfff640 10 librbd::image::CloseRequest: 0x563909e6cd90 handle_flush_image_watcher: r=0 2026-02-22T21:55:09.879 INFO:tasks.workunit.client.0.trial093.stdout:2026-02-22T21:55:09.870+0000 7f1c5bfff640 10 librbd::ImageState: 0x563909df7c20 0x563909df7c20 handle_close: r=0 2026-02-22T21:55:09.879 INFO:tasks.workunit.client.0.trial093.stdout:2026-02-22T21:55:09.870+0000 7f1c46ffd640 10 librbd::ImageCtx: 0x563909e1ba50 ~ImageCtx 2026-02-22T21:55:09.879 INFO:tasks.workunit.client.0.trial093.stdout:2026-02-22T21:55:09.870+0000 7f1c46ffd640 20 librbd::AsioEngine: 0x563909e6b0d0 ~AsioEngine: 2026-02-22T21:55:09.879 INFO:tasks.workunit.client.0.trial093.stdout:2026-02-22T21:55:09.870+0000 7f1c46ffd640 20 librbd::asio::ContextWQ: 0x563909e48910 ~ContextWQ: 2026-02-22T21:55:09.879 INFO:tasks.workunit.client.0.trial093.stdout:2026-02-22T21:55:09.870+0000 7f1c46ffd640 20 librbd::asio::ContextWQ: 0x563909e48910 drain: 2026-02-22T21:55:09.880 INFO:tasks.workunit.client.0.trial093.stderr:++ rbd_watch_fifo testimg17344 2026-02-22T21:55:09.881 INFO:tasks.workunit.client.0.trial093.stderr:++ echo /tmp/rbd_test_admin_socket17344/rbd_watch_testimg17344.fifo 2026-02-22T21:55:09.881 INFO:tasks.workunit.client.0.trial093.stderr:Terminated 2026-02-22T21:55:09.881 INFO:tasks.workunit.client.0.trial093.stderr:++ rbd_watch_pid_file testimg17344 2026-02-22T21:55:09.881 INFO:tasks.workunit.client.0.trial093.stderr:++ echo /tmp/rbd_test_admin_socket17344/rbd_watch_testimg17344.pid 2026-02-22T21:55:09.881 INFO:tasks.workunit.client.0.trial093.stderr:++ rbd_watch_out_file testimg17344 2026-02-22T21:55:09.881 INFO:tasks.workunit.client.0.trial093.stderr:++ echo /tmp/rbd_test_admin_socket17344/rbd_watch_testimg17344.out 2026-02-22T21:55:09.881 INFO:tasks.workunit.client.0.trial093.stderr:++ rbd_watch_asok testimg17344 2026-02-22T21:55:09.881 INFO:tasks.workunit.client.0.trial093.stderr:++ echo /tmp/rbd_test_admin_socket17344/rbd_watch_testimg17344.asok 2026-02-22T21:55:09.881 INFO:tasks.workunit.client.0.trial093.stderr:+ rm -f /tmp/rbd_test_admin_socket17344/rbd_watch_testimg17344.fifo /tmp/rbd_test_admin_socket17344/rbd_watch_testimg17344.pid /tmp/rbd_test_admin_socket17344/rbd_watch_testimg17344.out /tmp/rbd_test_admin_socket17344/rbd_watch_testimg17344.asok 2026-02-22T21:55:09.881 INFO:tasks.workunit.client.0.trial093.stderr:+ for conf_rbd_cache in false true 2026-02-22T21:55:09.882 INFO:tasks.workunit.client.0.trial093.stderr:+ rbd image-meta set testimg17344 conf_rbd_cache false 2026-02-22T21:55:09.922 INFO:tasks.workunit.client.0.trial093.stderr:+ rbd_watch_start testimg17344 2026-02-22T21:55:09.922 INFO:tasks.workunit.client.0.trial093.stderr:+ local image=testimg17344 2026-02-22T21:55:09.922 INFO:tasks.workunit.client.0.trial093.stderr:++ rbd_watch_asok testimg17344 2026-02-22T21:55:09.922 INFO:tasks.workunit.client.0.trial093.stderr:++ echo /tmp/rbd_test_admin_socket17344/rbd_watch_testimg17344.asok 2026-02-22T21:55:09.923 INFO:tasks.workunit.client.0.trial093.stderr:+ local asok=/tmp/rbd_test_admin_socket17344/rbd_watch_testimg17344.asok 2026-02-22T21:55:09.923 INFO:tasks.workunit.client.0.trial093.stderr:++ rbd_watch_fifo testimg17344 2026-02-22T21:55:09.923 INFO:tasks.workunit.client.0.trial093.stderr:++ echo /tmp/rbd_test_admin_socket17344/rbd_watch_testimg17344.fifo 2026-02-22T21:55:09.923 INFO:tasks.workunit.client.0.trial093.stderr:+ mkfifo /tmp/rbd_test_admin_socket17344/rbd_watch_testimg17344.fifo 2026-02-22T21:55:09.924 INFO:tasks.workunit.client.0.trial093.stderr:+ local pid 2026-02-22T21:55:09.925 INFO:tasks.workunit.client.0.trial093.stderr:+ rbd --admin-socket /tmp/rbd_test_admin_socket17344/rbd_watch_testimg17344.asok watch testimg17344 2026-02-22T21:55:09.925 INFO:tasks.workunit.client.0.trial093.stderr:++ seq 10 2026-02-22T21:55:09.925 INFO:tasks.workunit.client.0.trial093.stderr:++ rbd_watch_fifo testimg17344 2026-02-22T21:55:09.925 INFO:tasks.workunit.client.0.trial093.stderr:++ echo /tmp/rbd_test_admin_socket17344/rbd_watch_testimg17344.fifo 2026-02-22T21:55:09.926 INFO:tasks.workunit.client.0.trial093.stderr:++ rbd_watch_out_file testimg17344 2026-02-22T21:55:09.926 INFO:tasks.workunit.client.0.trial093.stderr:++ echo /tmp/rbd_test_admin_socket17344/rbd_watch_testimg17344.out 2026-02-22T21:55:09.926 INFO:tasks.workunit.client.0.trial093.stderr:+ cat /tmp/rbd_test_admin_socket17344/rbd_watch_testimg17344.fifo 2026-02-22T21:55:09.926 INFO:tasks.workunit.client.0.trial093.stderr:+ for i in `seq 10` 2026-02-22T21:55:09.927 INFO:tasks.workunit.client.0.trial093.stderr:++ ps auxww 2026-02-22T21:55:09.927 INFO:tasks.workunit.client.0.trial093.stderr:++ awk '/[r]bd --admin.* watch testimg17344/ {print $2}' 2026-02-22T21:55:09.938 INFO:tasks.workunit.client.0.trial093.stderr:+ pid=17467 2026-02-22T21:55:09.938 INFO:tasks.workunit.client.0.trial093.stderr:+ test -n 17467 2026-02-22T21:55:09.938 INFO:tasks.workunit.client.0.trial093.stderr:+ break 2026-02-22T21:55:09.938 INFO:tasks.workunit.client.0.trial093.stderr:+ test -n 17467 2026-02-22T21:55:09.938 INFO:tasks.workunit.client.0.trial093.stderr:+ echo 17467 2026-02-22T21:55:09.938 INFO:tasks.workunit.client.0.trial093.stderr:++ rbd_watch_pid_file testimg17344 2026-02-22T21:55:09.939 INFO:tasks.workunit.client.0.trial093.stderr:++ echo /tmp/rbd_test_admin_socket17344/rbd_watch_testimg17344.pid 2026-02-22T21:55:09.939 INFO:tasks.workunit.client.0.trial093.stderr:+ test -n /tmp/rbd_test_admin_socket17344/rbd_watch_testimg17344.asok 2026-02-22T21:55:09.939 INFO:tasks.workunit.client.0.trial093.stderr:++ seq 10 2026-02-22T21:55:09.940 INFO:tasks.workunit.client.0.trial093.stderr:+ for i in `seq 10` 2026-02-22T21:55:09.940 INFO:tasks.workunit.client.0.trial093.stderr:+ test -S /tmp/rbd_test_admin_socket17344/rbd_watch_testimg17344.asok 2026-02-22T21:55:09.940 INFO:tasks.workunit.client.0.trial093.stderr:+ sleep 0.1 2026-02-22T21:55:10.041 INFO:tasks.workunit.client.0.trial093.stderr:+ for i in `seq 10` 2026-02-22T21:55:10.041 INFO:tasks.workunit.client.0.trial093.stderr:+ test -S /tmp/rbd_test_admin_socket17344/rbd_watch_testimg17344.asok 2026-02-22T21:55:10.042 INFO:tasks.workunit.client.0.trial093.stderr:+ break 2026-02-22T21:55:10.042 INFO:tasks.workunit.client.0.trial093.stderr:+ test -S /tmp/rbd_test_admin_socket17344/rbd_watch_testimg17344.asok 2026-02-22T21:55:10.042 INFO:tasks.workunit.client.0.trial093.stderr:+ ceph --admin-daemon /tmp/rbd_test_admin_socket17344/rbd_watch_testimg17344.asok config set debug_rbd 20 2026-02-22T21:55:10.089 INFO:tasks.workunit.client.0.trial093.stdout:{ 2026-02-22T21:55:10.089 INFO:tasks.workunit.client.0.trial093.stdout: "success": "" 2026-02-22T21:55:10.089 INFO:tasks.workunit.client.0.trial093.stdout:} 2026-02-22T21:55:10.095 INFO:tasks.workunit.client.0.trial093.stderr:+ rbd status testimg17344 2026-02-22T21:55:10.095 INFO:tasks.workunit.client.0.trial093.stderr:+ expect_false grep 'Watchers: none' 2026-02-22T21:55:10.095 INFO:tasks.workunit.client.0.trial093.stderr:+ set -x 2026-02-22T21:55:10.095 INFO:tasks.workunit.client.0.trial093.stderr:+ grep 'Watchers: none' 2026-02-22T21:55:10.129 INFO:tasks.workunit.client.0.trial093.stderr:+ return 0 2026-02-22T21:55:10.129 INFO:tasks.workunit.client.0.trial093.stderr:+ rbd_check_perfcounter testimg17344 flush 0 2026-02-22T21:55:10.129 INFO:tasks.workunit.client.0.trial093.stderr:+ local image=testimg17344 2026-02-22T21:55:10.129 INFO:tasks.workunit.client.0.trial093.stderr:+ local counter=flush 2026-02-22T21:55:10.129 INFO:tasks.workunit.client.0.trial093.stderr:+ local expected_val=0 2026-02-22T21:55:10.129 INFO:tasks.workunit.client.0.trial093.stderr:+ local val= 2026-02-22T21:55:10.130 INFO:tasks.workunit.client.0.trial093.stderr:++ rbd_get_perfcounter testimg17344 flush 2026-02-22T21:55:10.130 INFO:tasks.workunit.client.0.trial093.stderr:++ local image=testimg17344 2026-02-22T21:55:10.130 INFO:tasks.workunit.client.0.trial093.stderr:++ local counter=flush 2026-02-22T21:55:10.130 INFO:tasks.workunit.client.0.trial093.stderr:++ local name 2026-02-22T21:55:10.131 INFO:tasks.workunit.client.0.trial093.stderr:+++ xmlstarlet el -d3 2026-02-22T21:55:10.131 INFO:tasks.workunit.client.0.trial093.stderr:+++ grep '/librbd-.*-testimg17344/flush$' 2026-02-22T21:55:10.131 INFO:tasks.workunit.client.0.trial093.stderr:++++ rbd_watch_asok testimg17344 2026-02-22T21:55:10.131 INFO:tasks.workunit.client.0.trial093.stderr:++++ echo /tmp/rbd_test_admin_socket17344/rbd_watch_testimg17344.asok 2026-02-22T21:55:10.131 INFO:tasks.workunit.client.0.trial093.stderr:+++ ceph --format xml --admin-daemon /tmp/rbd_test_admin_socket17344/rbd_watch_testimg17344.asok perf schema 2026-02-22T21:55:10.180 INFO:tasks.workunit.client.0.trial093.stderr:-:1.41: Failed to parse QName 'AsyncMessenger:' 2026-02-22T21:55:10.180 INFO:tasks.workunit.client.0.trial093.stderr:100none1106400100none1106400100none1106400100none1106400100none1109280100none1109280100none1109280100none1109280 sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2026-02-22T21:55:11.660 INFO:tasks.workunit:Stopping ['rbd/test_admin_socket.sh'] on client.0... 2026-02-22T21:55:11.660 DEBUG:teuthology.orchestra.run.trial093:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2026-02-22T21:55:11.970 DEBUG:teuthology.parallel:result is None 2026-02-22T21:55:11.970 DEBUG:teuthology.orchestra.run.trial093:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2026-02-22T21:55:11.979 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2026-02-22T21:55:11.979 DEBUG:teuthology.orchestra.run.trial093:> rmdir -- /home/ubuntu/cephtest/mnt.0 2026-02-22T21:55:12.024 INFO:tasks.workunit:Deleted artificial mount point /home/ubuntu/cephtest/mnt.0/client.0 2026-02-22T21:55:12.024 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2026-02-22T21:55:12.041 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2026-02-22T21:55:12.042 DEBUG:teuthology.orchestra.run.trial093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T21:55:12.222 INFO:teuthology.orchestra.run.trial093.stdout: 2026-02-22T21:55:12.223 INFO:teuthology.orchestra.run.trial093.stderr:dumped all 2026-02-22T21:55:12.233 INFO:teuthology.orchestra.run.trial093.stdout:{"pg_ready":true,"pg_map":{"version":32,"stamp":"2026-02-22T21:55:10.951882+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":56,"num_read_kb":46,"num_write":81,"num_write_kb":1108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":55,"ondisk_log_size":55,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":55352,"kb_used_data":1424,"kb_used_omap":3,"kb_used_meta":53884,"kb_avail":188688328,"statfs":{"total":193273528320,"available":193216847872,"internally_reserved":0,"allocated":1458176,"data_stored":1251768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":3188,"internal_metadata":55178124},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001412"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":21,"reported_epoch":15,"state":"active+clean","last_fresh":"2026-02-22T21:54:46.741968+0000","last_change":"2026-02-22T21:54:43.978642+0000","last_active":"2026-02-22T21:54:46.741968+0000","last_peered":"2026-02-22T21:54:46.741968+0000","last_clean":"2026-02-22T21:54:46.741968+0000","last_became_active":"2026-02-22T21:54:41.977824+0000","last_became_peered":"2026-02-22T21:54:41.977824+0000","last_unstale":"2026-02-22T21:54:46.741968+0000","last_undegraded":"2026-02-22T21:54:46.741968+0000","last_fullsized":"2026-02-22T21:54:46.741968+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T21:54:40.966951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T21:54:40.966951+0000","last_clean_scrub_stamp":"2026-02-22T21:54:40.966951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:31:00.749250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00078258400000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":21,"reported_epoch":15,"state":"active+clean","last_fresh":"2026-02-22T21:54:46.741964+0000","last_change":"2026-02-22T21:54:43.978636+0000","last_active":"2026-02-22T21:54:46.741964+0000","last_peered":"2026-02-22T21:54:46.741964+0000","last_clean":"2026-02-22T21:54:46.741964+0000","last_became_active":"2026-02-22T21:54:41.977649+0000","last_became_peered":"2026-02-22T21:54:41.977649+0000","last_unstale":"2026-02-22T21:54:46.741964+0000","last_undegraded":"2026-02-22T21:54:46.741964+0000","last_fullsized":"2026-02-22T21:54:46.741964+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T21:54:40.966951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T21:54:40.966951+0000","last_clean_scrub_stamp":"2026-02-22T21:54:40.966951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T01:28:56.435752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00078215399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":21,"reported_epoch":15,"state":"active+clean","last_fresh":"2026-02-22T21:54:46.741959+0000","last_change":"2026-02-22T21:54:43.978182+0000","last_active":"2026-02-22T21:54:46.741959+0000","last_peered":"2026-02-22T21:54:46.741959+0000","last_clean":"2026-02-22T21:54:46.741959+0000","last_became_active":"2026-02-22T21:54:41.977655+0000","last_became_peered":"2026-02-22T21:54:41.977655+0000","last_unstale":"2026-02-22T21:54:46.741959+0000","last_undegraded":"2026-02-22T21:54:46.741959+0000","last_fullsized":"2026-02-22T21:54:46.741959+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T21:54:40.966951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T21:54:40.966951+0000","last_clean_scrub_stamp":"2026-02-22T21:54:40.966951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:17:02.624184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00038385300000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":21,"reported_epoch":15,"state":"active+clean","last_fresh":"2026-02-22T21:54:46.741924+0000","last_change":"2026-02-22T21:54:43.978364+0000","last_active":"2026-02-22T21:54:46.741924+0000","last_peered":"2026-02-22T21:54:46.741924+0000","last_clean":"2026-02-22T21:54:46.741924+0000","last_became_active":"2026-02-22T21:54:41.977538+0000","last_became_peered":"2026-02-22T21:54:41.977538+0000","last_unstale":"2026-02-22T21:54:46.741924+0000","last_undegraded":"2026-02-22T21:54:46.741924+0000","last_fullsized":"2026-02-22T21:54:46.741924+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T21:54:40.966951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T21:54:40.966951+0000","last_clean_scrub_stamp":"2026-02-22T21:54:40.966951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T01:18:23.057694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057796400000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"14'2","reported_seq":23,"reported_epoch":15,"state":"active+clean","last_fresh":"2026-02-22T21:54:47.436860+0000","last_change":"2026-02-22T21:54:43.976742+0000","last_active":"2026-02-22T21:54:47.436860+0000","last_peered":"2026-02-22T21:54:47.436860+0000","last_clean":"2026-02-22T21:54:47.436860+0000","last_became_active":"2026-02-22T21:54:41.976693+0000","last_became_peered":"2026-02-22T21:54:41.976693+0000","last_unstale":"2026-02-22T21:54:47.436860+0000","last_undegraded":"2026-02-22T21:54:47.436860+0000","last_fullsized":"2026-02-22T21:54:47.436860+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T21:54:40.966951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T21:54:40.966951+0000","last_clean_scrub_stamp":"2026-02-22T21:54:40.966951+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T06:04:27.049416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057252400000000004,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":21,"reported_epoch":15,"state":"active+clean","last_fresh":"2026-02-22T21:54:46.741893+0000","last_change":"2026-02-22T21:54:43.978409+0000","last_active":"2026-02-22T21:54:46.741893+0000","last_peered":"2026-02-22T21:54:46.741893+0000","last_clean":"2026-02-22T21:54:46.741893+0000","last_became_active":"2026-02-22T21:54:41.977454+0000","last_became_peered":"2026-02-22T21:54:41.977454+0000","last_unstale":"2026-02-22T21:54:46.741893+0000","last_undegraded":"2026-02-22T21:54:46.741893+0000","last_fullsized":"2026-02-22T21:54:46.741893+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T21:54:40.966951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T21:54:40.966951+0000","last_clean_scrub_stamp":"2026-02-22T21:54:40.966951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:01:40.151823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000616426,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":21,"reported_epoch":15,"state":"active+clean","last_fresh":"2026-02-22T21:54:47.436829+0000","last_change":"2026-02-22T21:54:43.976467+0000","last_active":"2026-02-22T21:54:47.436829+0000","last_peered":"2026-02-22T21:54:47.436829+0000","last_clean":"2026-02-22T21:54:47.436829+0000","last_became_active":"2026-02-22T21:54:41.977088+0000","last_became_peered":"2026-02-22T21:54:41.977088+0000","last_unstale":"2026-02-22T21:54:47.436829+0000","last_undegraded":"2026-02-22T21:54:47.436829+0000","last_fullsized":"2026-02-22T21:54:47.436829+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T21:54:40.966951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T21:54:40.966951+0000","last_clean_scrub_stamp":"2026-02-22T21:54:40.966951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:11:36.719854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00028617200000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.3","version":"12'1","reported_seq":22,"reported_epoch":15,"state":"active+clean","last_fresh":"2026-02-22T21:54:46.741898+0000","last_change":"2026-02-22T21:54:43.978186+0000","last_active":"2026-02-22T21:54:46.741898+0000","last_peered":"2026-02-22T21:54:46.741898+0000","last_clean":"2026-02-22T21:54:46.741898+0000","last_became_active":"2026-02-22T21:54:41.977469+0000","last_became_peered":"2026-02-22T21:54:41.977469+0000","last_unstale":"2026-02-22T21:54:46.741898+0000","last_undegraded":"2026-02-22T21:54:46.741898+0000","last_fullsized":"2026-02-22T21:54:46.741898+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T21:54:40.966951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T21:54:40.966951+0000","last_clean_scrub_stamp":"2026-02-22T21:54:40.966951+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T09:17:09.429877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039984299999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"9'52","reported_seq":89,"reported_epoch":15,"state":"active+clean","last_fresh":"2026-02-22T21:54:46.741814+0000","last_change":"2026-02-22T21:54:37.359016+0000","last_active":"2026-02-22T21:54:46.741814+0000","last_peered":"2026-02-22T21:54:46.741814+0000","last_clean":"2026-02-22T21:54:46.741814+0000","last_became_active":"2026-02-22T21:54:37.358358+0000","last_became_peered":"2026-02-22T21:54:37.358358+0000","last_unstale":"2026-02-22T21:54:46.741814+0000","last_undegraded":"2026-02-22T21:54:46.741814+0000","last_fullsized":"2026-02-22T21:54:46.741814+0000","mapping_epoch":8,"log_start":"0'0","ondisk_log_start":"0'0","created":8,"last_epoch_clean":9,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T21:54:35.955267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T21:54:35.955267+0000","last_clean_scrub_stamp":"2026-02-22T21:54:35.955267+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:52:17.396301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":56,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":2},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":56,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":52,"ondisk_log_size":52,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":1,"up_from":7,"seq":30064771082,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27676,"kb_used_data":712,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344164,"statfs":{"total":96636764160,"available":96608423936,"internally_reserved":0,"allocated":729088,"data_stored":625884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27589062},"hb_peers":[0],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":7,"seq":30064771081,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27676,"kb_used_data":712,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344164,"statfs":{"total":96636764160,"available":96608423936,"internally_reserved":0,"allocated":729088,"data_stored":625884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27589062},"hb_peers":[1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T21:55:12.235 DEBUG:teuthology.orchestra.run.trial093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T21:55:12.386 INFO:teuthology.orchestra.run.trial093.stdout: 2026-02-22T21:55:12.386 INFO:teuthology.orchestra.run.trial093.stderr:dumped all 2026-02-22T21:55:12.396 INFO:teuthology.orchestra.run.trial093.stdout:{"pg_ready":true,"pg_map":{"version":32,"stamp":"2026-02-22T21:55:10.951882+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":56,"num_read_kb":46,"num_write":81,"num_write_kb":1108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":55,"ondisk_log_size":55,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":55352,"kb_used_data":1424,"kb_used_omap":3,"kb_used_meta":53884,"kb_avail":188688328,"statfs":{"total":193273528320,"available":193216847872,"internally_reserved":0,"allocated":1458176,"data_stored":1251768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":3188,"internal_metadata":55178124},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001412"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":21,"reported_epoch":15,"state":"active+clean","last_fresh":"2026-02-22T21:54:46.741968+0000","last_change":"2026-02-22T21:54:43.978642+0000","last_active":"2026-02-22T21:54:46.741968+0000","last_peered":"2026-02-22T21:54:46.741968+0000","last_clean":"2026-02-22T21:54:46.741968+0000","last_became_active":"2026-02-22T21:54:41.977824+0000","last_became_peered":"2026-02-22T21:54:41.977824+0000","last_unstale":"2026-02-22T21:54:46.741968+0000","last_undegraded":"2026-02-22T21:54:46.741968+0000","last_fullsized":"2026-02-22T21:54:46.741968+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T21:54:40.966951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T21:54:40.966951+0000","last_clean_scrub_stamp":"2026-02-22T21:54:40.966951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:31:00.749250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00078258400000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":21,"reported_epoch":15,"state":"active+clean","last_fresh":"2026-02-22T21:54:46.741964+0000","last_change":"2026-02-22T21:54:43.978636+0000","last_active":"2026-02-22T21:54:46.741964+0000","last_peered":"2026-02-22T21:54:46.741964+0000","last_clean":"2026-02-22T21:54:46.741964+0000","last_became_active":"2026-02-22T21:54:41.977649+0000","last_became_peered":"2026-02-22T21:54:41.977649+0000","last_unstale":"2026-02-22T21:54:46.741964+0000","last_undegraded":"2026-02-22T21:54:46.741964+0000","last_fullsized":"2026-02-22T21:54:46.741964+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T21:54:40.966951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T21:54:40.966951+0000","last_clean_scrub_stamp":"2026-02-22T21:54:40.966951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T01:28:56.435752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00078215399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":21,"reported_epoch":15,"state":"active+clean","last_fresh":"2026-02-22T21:54:46.741959+0000","last_change":"2026-02-22T21:54:43.978182+0000","last_active":"2026-02-22T21:54:46.741959+0000","last_peered":"2026-02-22T21:54:46.741959+0000","last_clean":"2026-02-22T21:54:46.741959+0000","last_became_active":"2026-02-22T21:54:41.977655+0000","last_became_peered":"2026-02-22T21:54:41.977655+0000","last_unstale":"2026-02-22T21:54:46.741959+0000","last_undegraded":"2026-02-22T21:54:46.741959+0000","last_fullsized":"2026-02-22T21:54:46.741959+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T21:54:40.966951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T21:54:40.966951+0000","last_clean_scrub_stamp":"2026-02-22T21:54:40.966951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:17:02.624184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00038385300000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":21,"reported_epoch":15,"state":"active+clean","last_fresh":"2026-02-22T21:54:46.741924+0000","last_change":"2026-02-22T21:54:43.978364+0000","last_active":"2026-02-22T21:54:46.741924+0000","last_peered":"2026-02-22T21:54:46.741924+0000","last_clean":"2026-02-22T21:54:46.741924+0000","last_became_active":"2026-02-22T21:54:41.977538+0000","last_became_peered":"2026-02-22T21:54:41.977538+0000","last_unstale":"2026-02-22T21:54:46.741924+0000","last_undegraded":"2026-02-22T21:54:46.741924+0000","last_fullsized":"2026-02-22T21:54:46.741924+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T21:54:40.966951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T21:54:40.966951+0000","last_clean_scrub_stamp":"2026-02-22T21:54:40.966951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T01:18:23.057694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057796400000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"14'2","reported_seq":23,"reported_epoch":15,"state":"active+clean","last_fresh":"2026-02-22T21:54:47.436860+0000","last_change":"2026-02-22T21:54:43.976742+0000","last_active":"2026-02-22T21:54:47.436860+0000","last_peered":"2026-02-22T21:54:47.436860+0000","last_clean":"2026-02-22T21:54:47.436860+0000","last_became_active":"2026-02-22T21:54:41.976693+0000","last_became_peered":"2026-02-22T21:54:41.976693+0000","last_unstale":"2026-02-22T21:54:47.436860+0000","last_undegraded":"2026-02-22T21:54:47.436860+0000","last_fullsized":"2026-02-22T21:54:47.436860+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T21:54:40.966951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T21:54:40.966951+0000","last_clean_scrub_stamp":"2026-02-22T21:54:40.966951+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T06:04:27.049416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057252400000000004,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":21,"reported_epoch":15,"state":"active+clean","last_fresh":"2026-02-22T21:54:46.741893+0000","last_change":"2026-02-22T21:54:43.978409+0000","last_active":"2026-02-22T21:54:46.741893+0000","last_peered":"2026-02-22T21:54:46.741893+0000","last_clean":"2026-02-22T21:54:46.741893+0000","last_became_active":"2026-02-22T21:54:41.977454+0000","last_became_peered":"2026-02-22T21:54:41.977454+0000","last_unstale":"2026-02-22T21:54:46.741893+0000","last_undegraded":"2026-02-22T21:54:46.741893+0000","last_fullsized":"2026-02-22T21:54:46.741893+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T21:54:40.966951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T21:54:40.966951+0000","last_clean_scrub_stamp":"2026-02-22T21:54:40.966951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:01:40.151823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000616426,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":21,"reported_epoch":15,"state":"active+clean","last_fresh":"2026-02-22T21:54:47.436829+0000","last_change":"2026-02-22T21:54:43.976467+0000","last_active":"2026-02-22T21:54:47.436829+0000","last_peered":"2026-02-22T21:54:47.436829+0000","last_clean":"2026-02-22T21:54:47.436829+0000","last_became_active":"2026-02-22T21:54:41.977088+0000","last_became_peered":"2026-02-22T21:54:41.977088+0000","last_unstale":"2026-02-22T21:54:47.436829+0000","last_undegraded":"2026-02-22T21:54:47.436829+0000","last_fullsized":"2026-02-22T21:54:47.436829+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T21:54:40.966951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T21:54:40.966951+0000","last_clean_scrub_stamp":"2026-02-22T21:54:40.966951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:11:36.719854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00028617200000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.3","version":"12'1","reported_seq":22,"reported_epoch":15,"state":"active+clean","last_fresh":"2026-02-22T21:54:46.741898+0000","last_change":"2026-02-22T21:54:43.978186+0000","last_active":"2026-02-22T21:54:46.741898+0000","last_peered":"2026-02-22T21:54:46.741898+0000","last_clean":"2026-02-22T21:54:46.741898+0000","last_became_active":"2026-02-22T21:54:41.977469+0000","last_became_peered":"2026-02-22T21:54:41.977469+0000","last_unstale":"2026-02-22T21:54:46.741898+0000","last_undegraded":"2026-02-22T21:54:46.741898+0000","last_fullsized":"2026-02-22T21:54:46.741898+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T21:54:40.966951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T21:54:40.966951+0000","last_clean_scrub_stamp":"2026-02-22T21:54:40.966951+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T09:17:09.429877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039984299999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"9'52","reported_seq":89,"reported_epoch":15,"state":"active+clean","last_fresh":"2026-02-22T21:54:46.741814+0000","last_change":"2026-02-22T21:54:37.359016+0000","last_active":"2026-02-22T21:54:46.741814+0000","last_peered":"2026-02-22T21:54:46.741814+0000","last_clean":"2026-02-22T21:54:46.741814+0000","last_became_active":"2026-02-22T21:54:37.358358+0000","last_became_peered":"2026-02-22T21:54:37.358358+0000","last_unstale":"2026-02-22T21:54:46.741814+0000","last_undegraded":"2026-02-22T21:54:46.741814+0000","last_fullsized":"2026-02-22T21:54:46.741814+0000","mapping_epoch":8,"log_start":"0'0","ondisk_log_start":"0'0","created":8,"last_epoch_clean":9,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T21:54:35.955267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T21:54:35.955267+0000","last_clean_scrub_stamp":"2026-02-22T21:54:35.955267+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:52:17.396301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":56,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":2},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":56,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":52,"ondisk_log_size":52,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":1,"up_from":7,"seq":30064771082,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27676,"kb_used_data":712,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344164,"statfs":{"total":96636764160,"available":96608423936,"internally_reserved":0,"allocated":729088,"data_stored":625884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27589062},"hb_peers":[0],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":7,"seq":30064771081,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27676,"kb_used_data":712,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344164,"statfs":{"total":96636764160,"available":96608423936,"internally_reserved":0,"allocated":729088,"data_stored":625884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27589062},"hb_peers":[1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T21:55:12.396 INFO:tasks.ceph.ceph_manager.ceph:clean! 2026-02-22T21:55:12.396 DEBUG:teuthology.orchestra.run.trial093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T21:55:12.542 INFO:teuthology.orchestra.run.trial093.stdout: 2026-02-22T21:55:12.543 INFO:teuthology.orchestra.run.trial093.stderr:dumped all 2026-02-22T21:55:12.553 INFO:teuthology.orchestra.run.trial093.stdout:{"pg_ready":true,"pg_map":{"version":32,"stamp":"2026-02-22T21:55:10.951882+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":56,"num_read_kb":46,"num_write":81,"num_write_kb":1108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":55,"ondisk_log_size":55,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":55352,"kb_used_data":1424,"kb_used_omap":3,"kb_used_meta":53884,"kb_avail":188688328,"statfs":{"total":193273528320,"available":193216847872,"internally_reserved":0,"allocated":1458176,"data_stored":1251768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":3188,"internal_metadata":55178124},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001412"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":21,"reported_epoch":15,"state":"active+clean","last_fresh":"2026-02-22T21:54:46.741968+0000","last_change":"2026-02-22T21:54:43.978642+0000","last_active":"2026-02-22T21:54:46.741968+0000","last_peered":"2026-02-22T21:54:46.741968+0000","last_clean":"2026-02-22T21:54:46.741968+0000","last_became_active":"2026-02-22T21:54:41.977824+0000","last_became_peered":"2026-02-22T21:54:41.977824+0000","last_unstale":"2026-02-22T21:54:46.741968+0000","last_undegraded":"2026-02-22T21:54:46.741968+0000","last_fullsized":"2026-02-22T21:54:46.741968+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T21:54:40.966951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T21:54:40.966951+0000","last_clean_scrub_stamp":"2026-02-22T21:54:40.966951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:31:00.749250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00078258400000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":21,"reported_epoch":15,"state":"active+clean","last_fresh":"2026-02-22T21:54:46.741964+0000","last_change":"2026-02-22T21:54:43.978636+0000","last_active":"2026-02-22T21:54:46.741964+0000","last_peered":"2026-02-22T21:54:46.741964+0000","last_clean":"2026-02-22T21:54:46.741964+0000","last_became_active":"2026-02-22T21:54:41.977649+0000","last_became_peered":"2026-02-22T21:54:41.977649+0000","last_unstale":"2026-02-22T21:54:46.741964+0000","last_undegraded":"2026-02-22T21:54:46.741964+0000","last_fullsized":"2026-02-22T21:54:46.741964+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T21:54:40.966951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T21:54:40.966951+0000","last_clean_scrub_stamp":"2026-02-22T21:54:40.966951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T01:28:56.435752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00078215399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":21,"reported_epoch":15,"state":"active+clean","last_fresh":"2026-02-22T21:54:46.741959+0000","last_change":"2026-02-22T21:54:43.978182+0000","last_active":"2026-02-22T21:54:46.741959+0000","last_peered":"2026-02-22T21:54:46.741959+0000","last_clean":"2026-02-22T21:54:46.741959+0000","last_became_active":"2026-02-22T21:54:41.977655+0000","last_became_peered":"2026-02-22T21:54:41.977655+0000","last_unstale":"2026-02-22T21:54:46.741959+0000","last_undegraded":"2026-02-22T21:54:46.741959+0000","last_fullsized":"2026-02-22T21:54:46.741959+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T21:54:40.966951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T21:54:40.966951+0000","last_clean_scrub_stamp":"2026-02-22T21:54:40.966951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:17:02.624184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00038385300000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":21,"reported_epoch":15,"state":"active+clean","last_fresh":"2026-02-22T21:54:46.741924+0000","last_change":"2026-02-22T21:54:43.978364+0000","last_active":"2026-02-22T21:54:46.741924+0000","last_peered":"2026-02-22T21:54:46.741924+0000","last_clean":"2026-02-22T21:54:46.741924+0000","last_became_active":"2026-02-22T21:54:41.977538+0000","last_became_peered":"2026-02-22T21:54:41.977538+0000","last_unstale":"2026-02-22T21:54:46.741924+0000","last_undegraded":"2026-02-22T21:54:46.741924+0000","last_fullsized":"2026-02-22T21:54:46.741924+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T21:54:40.966951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T21:54:40.966951+0000","last_clean_scrub_stamp":"2026-02-22T21:54:40.966951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T01:18:23.057694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057796400000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"14'2","reported_seq":23,"reported_epoch":15,"state":"active+clean","last_fresh":"2026-02-22T21:54:47.436860+0000","last_change":"2026-02-22T21:54:43.976742+0000","last_active":"2026-02-22T21:54:47.436860+0000","last_peered":"2026-02-22T21:54:47.436860+0000","last_clean":"2026-02-22T21:54:47.436860+0000","last_became_active":"2026-02-22T21:54:41.976693+0000","last_became_peered":"2026-02-22T21:54:41.976693+0000","last_unstale":"2026-02-22T21:54:47.436860+0000","last_undegraded":"2026-02-22T21:54:47.436860+0000","last_fullsized":"2026-02-22T21:54:47.436860+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T21:54:40.966951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T21:54:40.966951+0000","last_clean_scrub_stamp":"2026-02-22T21:54:40.966951+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T06:04:27.049416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057252400000000004,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":21,"reported_epoch":15,"state":"active+clean","last_fresh":"2026-02-22T21:54:46.741893+0000","last_change":"2026-02-22T21:54:43.978409+0000","last_active":"2026-02-22T21:54:46.741893+0000","last_peered":"2026-02-22T21:54:46.741893+0000","last_clean":"2026-02-22T21:54:46.741893+0000","last_became_active":"2026-02-22T21:54:41.977454+0000","last_became_peered":"2026-02-22T21:54:41.977454+0000","last_unstale":"2026-02-22T21:54:46.741893+0000","last_undegraded":"2026-02-22T21:54:46.741893+0000","last_fullsized":"2026-02-22T21:54:46.741893+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T21:54:40.966951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T21:54:40.966951+0000","last_clean_scrub_stamp":"2026-02-22T21:54:40.966951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:01:40.151823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000616426,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":21,"reported_epoch":15,"state":"active+clean","last_fresh":"2026-02-22T21:54:47.436829+0000","last_change":"2026-02-22T21:54:43.976467+0000","last_active":"2026-02-22T21:54:47.436829+0000","last_peered":"2026-02-22T21:54:47.436829+0000","last_clean":"2026-02-22T21:54:47.436829+0000","last_became_active":"2026-02-22T21:54:41.977088+0000","last_became_peered":"2026-02-22T21:54:41.977088+0000","last_unstale":"2026-02-22T21:54:47.436829+0000","last_undegraded":"2026-02-22T21:54:47.436829+0000","last_fullsized":"2026-02-22T21:54:47.436829+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T21:54:40.966951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T21:54:40.966951+0000","last_clean_scrub_stamp":"2026-02-22T21:54:40.966951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:11:36.719854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00028617200000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.3","version":"12'1","reported_seq":22,"reported_epoch":15,"state":"active+clean","last_fresh":"2026-02-22T21:54:46.741898+0000","last_change":"2026-02-22T21:54:43.978186+0000","last_active":"2026-02-22T21:54:46.741898+0000","last_peered":"2026-02-22T21:54:46.741898+0000","last_clean":"2026-02-22T21:54:46.741898+0000","last_became_active":"2026-02-22T21:54:41.977469+0000","last_became_peered":"2026-02-22T21:54:41.977469+0000","last_unstale":"2026-02-22T21:54:46.741898+0000","last_undegraded":"2026-02-22T21:54:46.741898+0000","last_fullsized":"2026-02-22T21:54:46.741898+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T21:54:40.966951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T21:54:40.966951+0000","last_clean_scrub_stamp":"2026-02-22T21:54:40.966951+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T09:17:09.429877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039984299999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"9'52","reported_seq":89,"reported_epoch":15,"state":"active+clean","last_fresh":"2026-02-22T21:54:46.741814+0000","last_change":"2026-02-22T21:54:37.359016+0000","last_active":"2026-02-22T21:54:46.741814+0000","last_peered":"2026-02-22T21:54:46.741814+0000","last_clean":"2026-02-22T21:54:46.741814+0000","last_became_active":"2026-02-22T21:54:37.358358+0000","last_became_peered":"2026-02-22T21:54:37.358358+0000","last_unstale":"2026-02-22T21:54:46.741814+0000","last_undegraded":"2026-02-22T21:54:46.741814+0000","last_fullsized":"2026-02-22T21:54:46.741814+0000","mapping_epoch":8,"log_start":"0'0","ondisk_log_start":"0'0","created":8,"last_epoch_clean":9,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T21:54:35.955267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T21:54:35.955267+0000","last_clean_scrub_stamp":"2026-02-22T21:54:35.955267+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:52:17.396301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":56,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":2},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":56,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":52,"ondisk_log_size":52,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":1,"up_from":7,"seq":30064771082,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27676,"kb_used_data":712,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344164,"statfs":{"total":96636764160,"available":96608423936,"internally_reserved":0,"allocated":729088,"data_stored":625884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27589062},"hb_peers":[0],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":7,"seq":30064771081,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27676,"kb_used_data":712,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344164,"statfs":{"total":96636764160,"available":96608423936,"internally_reserved":0,"allocated":729088,"data_stored":625884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27589062},"hb_peers":[1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T21:55:12.554 DEBUG:teuthology.orchestra.run.trial093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2026-02-22T21:55:12.701 INFO:teuthology.orchestra.run.trial093.stdout: 2026-02-22T21:55:12.702 INFO:teuthology.orchestra.run.trial093.stdout:{"epoch":15,"fsid":"63f41746-3baf-462b-9374-bd9179f37750","created":"2026-02-22T21:54:26.784109+0000","modified":"2026-02-22T21:54:46.739795+0000","last_up_change":"2026-02-22T21:54:34.951859+0000","last_in_change":"2026-02-22T21:54:27.145522+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":2,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2026-02-22T21:54:34.979937+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"10","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2026-02-22T21:54:40.438651+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"14","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":14,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"7161d54e-1b56-4091-a2ca-0b42ae8b3475","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":7,"up_thru":11,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.93:6800","nonce":1805491287},{"type":"v1","addr":"10.20.193.93:6801","nonce":1805491287}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.93:6802","nonce":1805491287},{"type":"v1","addr":"10.20.193.93:6803","nonce":1805491287}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.93:6806","nonce":1805491287},{"type":"v1","addr":"10.20.193.93:6807","nonce":1805491287}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.93:6804","nonce":1805491287},{"type":"v1","addr":"10.20.193.93:6805","nonce":1805491287}]},"public_addr":"10.20.193.93:6801/1805491287","cluster_addr":"10.20.193.93:6803/1805491287","heartbeat_back_addr":"10.20.193.93:6807/1805491287","heartbeat_front_addr":"10.20.193.93:6805/1805491287","state":["exists","up"]},{"osd":1,"uuid":"a7ea7aed-6c59-4184-8da7-354423cced23","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":7,"up_thru":11,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.93:6808","nonce":1536829300},{"type":"v1","addr":"10.20.193.93:6809","nonce":1536829300}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.93:6810","nonce":1536829300},{"type":"v1","addr":"10.20.193.93:6811","nonce":1536829300}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.93:6814","nonce":1536829300},{"type":"v1","addr":"10.20.193.93:6815","nonce":1536829300}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.93:6812","nonce":1536829300},{"type":"v1","addr":"10.20.193.93:6813","nonce":1536829300}]},"public_addr":"10.20.193.93:6809/1536829300","cluster_addr":"10.20.193.93:6811/1536829300","heartbeat_back_addr":"10.20.193.93:6815/1536829300","heartbeat_front_addr":"10.20.193.93:6813/1536829300","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2026-02-22T21:55:13.713 INFO:tasks.ceph:Scrubbing osd.0 2026-02-22T21:55:13.713 DEBUG:teuthology.orchestra.run.trial093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2026-02-22T21:55:13.797 INFO:teuthology.orchestra.run.trial093.stdout:{ 2026-02-22T21:55:13.798 INFO:teuthology.orchestra.run.trial093.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) osd_delete_sleep = '' osd_delete_sleep_hdd = '' osd_delete_sleep_hybrid = '' osd_delete_sleep_ssd = '' osd_max_backfills = '' osd_recovery_max_active = '' osd_recovery_max_active_hdd = '' osd_recovery_max_active_ssd = '' osd_recovery_sleep = '' osd_recovery_sleep_degraded = '' osd_recovery_sleep_degraded_hdd = '' osd_recovery_sleep_degraded_hybrid = '' osd_recovery_sleep_degraded_ssd = '' osd_recovery_sleep_hdd = '' osd_recovery_sleep_hybrid = '' osd_recovery_sleep_ssd = '' osd_scrub_sleep = '' osd_snap_trim_sleep = '' osd_snap_trim_sleep_hdd = '' osd_snap_trim_sleep_hybrid = '' osd_snap_trim_sleep_ssd = '' " 2026-02-22T21:55:13.798 INFO:teuthology.orchestra.run.trial093.stdout:} 2026-02-22T21:55:13.807 DEBUG:teuthology.orchestra.run.trial093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2026-02-22T21:55:13.953 INFO:teuthology.orchestra.run.trial093.stderr:instructed osd(s) 0 to deep-scrub 2026-02-22T21:55:13.964 INFO:tasks.ceph:Scrubbing osd.1 2026-02-22T21:55:13.965 DEBUG:teuthology.orchestra.run.trial093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2026-02-22T21:55:14.045 INFO:teuthology.orchestra.run.trial093.stdout:{ 2026-02-22T21:55:14.046 INFO:teuthology.orchestra.run.trial093.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) osd_delete_sleep = '' osd_delete_sleep_hdd = '' osd_delete_sleep_hybrid = '' osd_delete_sleep_ssd = '' osd_max_backfills = '' osd_recovery_max_active = '' osd_recovery_max_active_hdd = '' osd_recovery_max_active_ssd = '' osd_recovery_sleep = '' osd_recovery_sleep_degraded = '' osd_recovery_sleep_degraded_hdd = '' osd_recovery_sleep_degraded_hybrid = '' osd_recovery_sleep_degraded_ssd = '' osd_recovery_sleep_hdd = '' osd_recovery_sleep_hybrid = '' osd_recovery_sleep_ssd = '' osd_scrub_sleep = '' osd_snap_trim_sleep = '' osd_snap_trim_sleep_hdd = '' osd_snap_trim_sleep_hybrid = '' osd_snap_trim_sleep_ssd = '' " 2026-02-22T21:55:14.046 INFO:teuthology.orchestra.run.trial093.stdout:} 2026-02-22T21:55:14.055 DEBUG:teuthology.orchestra.run.trial093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2026-02-22T21:55:14.210 INFO:teuthology.orchestra.run.trial093.stderr:instructed osd(s) 1 to deep-scrub 2026-02-22T21:55:14.221 DEBUG:teuthology.orchestra.run.trial093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T21:55:14.369 INFO:teuthology.orchestra.run.trial093.stdout: 2026-02-22T21:55:14.369 INFO:teuthology.orchestra.run.trial093.stderr:dumped all 2026-02-22T21:55:14.381 INFO:teuthology.orchestra.run.trial093.stdout:{"pg_ready":true,"pg_map":{"version":33,"stamp":"2026-02-22T21:55:12.952069+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590445,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":310,"num_read_kb":255,"num_write":95,"num_write_kb":1114,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":78,"ondisk_log_size":78,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":55360,"kb_used_data":1440,"kb_used_omap":3,"kb_used_meta":53884,"kb_avail":188688320,"statfs":{"total":193273528320,"available":193216839680,"internally_reserved":0,"allocated":1474560,"data_stored":1251884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":3188,"internal_metadata":55178124},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":58,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":254,"num_read_kb":209,"num_write":14,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001359"},"pg_stats":[{"pgid":"2.7","version":"15'16","reported_seq":117,"reported_epoch":15,"state":"active+clean","last_fresh":"2026-02-22T21:55:10.781727+0000","last_change":"2026-02-22T21:54:43.978642+0000","last_active":"2026-02-22T21:55:10.781727+0000","last_peered":"2026-02-22T21:55:10.781727+0000","last_clean":"2026-02-22T21:55:10.781727+0000","last_became_active":"2026-02-22T21:54:41.977824+0000","last_became_peered":"2026-02-22T21:54:41.977824+0000","last_unstale":"2026-02-22T21:55:10.781727+0000","last_undegraded":"2026-02-22T21:55:10.781727+0000","last_fullsized":"2026-02-22T21:55:10.781727+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T21:54:40.966951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T21:54:40.966951+0000","last_clean_scrub_stamp":"2026-02-22T21:54:40.966951+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:31:00.749250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00078258400000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":209,"num_read_kb":184,"num_write":7,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":21,"reported_epoch":15,"state":"active+clean","last_fresh":"2026-02-22T21:54:46.741964+0000","last_change":"2026-02-22T21:54:43.978636+0000","last_active":"2026-02-22T21:54:46.741964+0000","last_peered":"2026-02-22T21:54:46.741964+0000","last_clean":"2026-02-22T21:54:46.741964+0000","last_became_active":"2026-02-22T21:54:41.977649+0000","last_became_peered":"2026-02-22T21:54:41.977649+0000","last_unstale":"2026-02-22T21:54:46.741964+0000","last_undegraded":"2026-02-22T21:54:46.741964+0000","last_fullsized":"2026-02-22T21:54:46.741964+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T21:54:40.966951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T21:54:40.966951+0000","last_clean_scrub_stamp":"2026-02-22T21:54:40.966951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T01:28:56.435752+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00078215399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":21,"reported_epoch":15,"state":"active+clean","last_fresh":"2026-02-22T21:54:46.741959+0000","last_change":"2026-02-22T21:54:43.978182+0000","last_active":"2026-02-22T21:54:46.741959+0000","last_peered":"2026-02-22T21:54:46.741959+0000","last_clean":"2026-02-22T21:54:46.741959+0000","last_became_active":"2026-02-22T21:54:41.977655+0000","last_became_peered":"2026-02-22T21:54:41.977655+0000","last_unstale":"2026-02-22T21:54:46.741959+0000","last_undegraded":"2026-02-22T21:54:46.741959+0000","last_fullsized":"2026-02-22T21:54:46.741959+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T21:54:40.966951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T21:54:40.966951+0000","last_clean_scrub_stamp":"2026-02-22T21:54:40.966951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:17:02.624184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00038385300000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"15'2","reported_seq":38,"reported_epoch":15,"state":"active+clean","last_fresh":"2026-02-22T21:55:10.779185+0000","last_change":"2026-02-22T21:54:43.978364+0000","last_active":"2026-02-22T21:55:10.779185+0000","last_peered":"2026-02-22T21:55:10.779185+0000","last_clean":"2026-02-22T21:55:10.779185+0000","last_became_active":"2026-02-22T21:54:41.977538+0000","last_became_peered":"2026-02-22T21:54:41.977538+0000","last_unstale":"2026-02-22T21:55:10.779185+0000","last_undegraded":"2026-02-22T21:55:10.779185+0000","last_fullsized":"2026-02-22T21:55:10.779185+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T21:54:40.966951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T21:54:40.966951+0000","last_clean_scrub_stamp":"2026-02-22T21:54:40.966951+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T01:18:23.057694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057796400000000004,"stat_sum":{"num_bytes":16,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":25,"num_read_kb":9,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"14'2","reported_seq":34,"reported_epoch":15,"state":"active+clean","last_fresh":"2026-02-22T21:55:10.751076+0000","last_change":"2026-02-22T21:54:43.976742+0000","last_active":"2026-02-22T21:55:10.751076+0000","last_peered":"2026-02-22T21:55:10.751076+0000","last_clean":"2026-02-22T21:55:10.751076+0000","last_became_active":"2026-02-22T21:54:41.976693+0000","last_became_peered":"2026-02-22T21:54:41.976693+0000","last_unstale":"2026-02-22T21:55:10.751076+0000","last_undegraded":"2026-02-22T21:55:10.751076+0000","last_fullsized":"2026-02-22T21:55:10.751076+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T21:54:40.966951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T21:54:40.966951+0000","last_clean_scrub_stamp":"2026-02-22T21:54:40.966951+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T06:04:27.049416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00057252400000000004,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":11,"num_read_kb":11,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"15'5","reported_seq":28,"reported_epoch":15,"state":"active+clean","last_fresh":"2026-02-22T21:55:10.749027+0000","last_change":"2026-02-22T21:54:43.978409+0000","last_active":"2026-02-22T21:55:10.749027+0000","last_peered":"2026-02-22T21:55:10.749027+0000","last_clean":"2026-02-22T21:55:10.749027+0000","last_became_active":"2026-02-22T21:54:41.977454+0000","last_became_peered":"2026-02-22T21:54:41.977454+0000","last_unstale":"2026-02-22T21:55:10.749027+0000","last_undegraded":"2026-02-22T21:55:10.749027+0000","last_fullsized":"2026-02-22T21:55:10.749027+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T21:54:40.966951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T21:54:40.966951+0000","last_clean_scrub_stamp":"2026-02-22T21:54:40.966951+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:01:40.151823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000616426,"stat_sum":{"num_bytes":42,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":9,"num_read_kb":5,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":21,"reported_epoch":15,"state":"active+clean","last_fresh":"2026-02-22T21:54:47.436829+0000","last_change":"2026-02-22T21:54:43.976467+0000","last_active":"2026-02-22T21:54:47.436829+0000","last_peered":"2026-02-22T21:54:47.436829+0000","last_clean":"2026-02-22T21:54:47.436829+0000","last_became_active":"2026-02-22T21:54:41.977088+0000","last_became_peered":"2026-02-22T21:54:41.977088+0000","last_unstale":"2026-02-22T21:54:47.436829+0000","last_undegraded":"2026-02-22T21:54:47.436829+0000","last_fullsized":"2026-02-22T21:54:47.436829+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T21:54:40.966951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T21:54:40.966951+0000","last_clean_scrub_stamp":"2026-02-22T21:54:40.966951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:11:36.719854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00028617200000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.3","version":"12'1","reported_seq":22,"reported_epoch":15,"state":"active+clean","last_fresh":"2026-02-22T21:54:46.741898+0000","last_change":"2026-02-22T21:54:43.978186+0000","last_active":"2026-02-22T21:54:46.741898+0000","last_peered":"2026-02-22T21:54:46.741898+0000","last_clean":"2026-02-22T21:54:46.741898+0000","last_became_active":"2026-02-22T21:54:41.977469+0000","last_became_peered":"2026-02-22T21:54:41.977469+0000","last_unstale":"2026-02-22T21:54:46.741898+0000","last_undegraded":"2026-02-22T21:54:46.741898+0000","last_fullsized":"2026-02-22T21:54:46.741898+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T21:54:40.966951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T21:54:40.966951+0000","last_clean_scrub_stamp":"2026-02-22T21:54:40.966951+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T09:17:09.429877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039984299999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"9'52","reported_seq":89,"reported_epoch":15,"state":"active+clean","last_fresh":"2026-02-22T21:54:46.741814+0000","last_change":"2026-02-22T21:54:37.359016+0000","last_active":"2026-02-22T21:54:46.741814+0000","last_peered":"2026-02-22T21:54:46.741814+0000","last_clean":"2026-02-22T21:54:46.741814+0000","last_became_active":"2026-02-22T21:54:37.358358+0000","last_became_peered":"2026-02-22T21:54:37.358358+0000","last_unstale":"2026-02-22T21:54:46.741814+0000","last_undegraded":"2026-02-22T21:54:46.741814+0000","last_fullsized":"2026-02-22T21:54:46.741814+0000","mapping_epoch":8,"log_start":"0'0","ondisk_log_start":"0'0","created":8,"last_epoch_clean":9,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T21:54:35.955267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T21:54:35.955267+0000","last_clean_scrub_stamp":"2026-02-22T21:54:35.955267+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:52:17.396301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":56,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":77,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":254,"num_read_kb":209,"num_write":16,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":154,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":26,"ondisk_log_size":26,"up":16,"acting":16,"num_store_stats":2},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":56,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":52,"ondisk_log_size":52,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":1,"up_from":7,"seq":30064771083,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27680,"kb_used_data":720,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344160,"statfs":{"total":96636764160,"available":96608419840,"internally_reserved":0,"allocated":737280,"data_stored":625942,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27589062},"hb_peers":[0],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":7,"seq":30064771082,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27680,"kb_used_data":720,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344160,"statfs":{"total":96636764160,"available":96608419840,"internally_reserved":0,"allocated":737280,"data_stored":625942,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27589062},"hb_peers":[1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":77,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":77,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T21:55:14.386 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2026-02-22T21:54:40.966951+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=22, tm_hour=21, tm_min=54, tm_sec=40, tm_wday=6, tm_yday=53, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=22, tm_hour=21, tm_min=55, tm_sec=12, tm_wday=6, tm_yday=53, tm_isdst=0) 2026-02-22T21:55:14.387 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2026-02-22T21:54:40.966951+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=22, tm_hour=21, tm_min=54, tm_sec=40, tm_wday=6, tm_yday=53, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=22, tm_hour=21, tm_min=55, tm_sec=12, tm_wday=6, tm_yday=53, tm_isdst=0) 2026-02-22T21:55:14.387 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2026-02-22T21:54:40.966951+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=22, tm_hour=21, tm_min=54, tm_sec=40, tm_wday=6, tm_yday=53, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=22, tm_hour=21, tm_min=55, tm_sec=12, tm_wday=6, tm_yday=53, tm_isdst=0) 2026-02-22T21:55:14.387 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2026-02-22T21:54:40.966951+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=22, tm_hour=21, tm_min=54, tm_sec=40, tm_wday=6, tm_yday=53, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=22, tm_hour=21, tm_min=55, tm_sec=12, tm_wday=6, tm_yday=53, tm_isdst=0) 2026-02-22T21:55:14.387 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2026-02-22T21:54:40.966951+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=22, tm_hour=21, tm_min=54, tm_sec=40, tm_wday=6, tm_yday=53, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=22, tm_hour=21, tm_min=55, tm_sec=12, tm_wday=6, tm_yday=53, tm_isdst=0) 2026-02-22T21:55:14.387 INFO:tasks.ceph:pgid 2.1 last_scrub_stamp 2026-02-22T21:54:40.966951+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=22, tm_hour=21, tm_min=54, tm_sec=40, tm_wday=6, tm_yday=53, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=22, tm_hour=21, tm_min=55, tm_sec=12, tm_wday=6, tm_yday=53, tm_isdst=0) 2026-02-22T21:55:14.387 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2026-02-22T21:54:40.966951+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=22, tm_hour=21, tm_min=54, tm_sec=40, tm_wday=6, tm_yday=53, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=22, tm_hour=21, tm_min=55, tm_sec=12, tm_wday=6, tm_yday=53, tm_isdst=0) 2026-02-22T21:55:14.388 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2026-02-22T21:54:40.966951+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=22, tm_hour=21, tm_min=54, tm_sec=40, tm_wday=6, tm_yday=53, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=22, tm_hour=21, tm_min=55, tm_sec=12, tm_wday=6, tm_yday=53, tm_isdst=0) 2026-02-22T21:55:14.388 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2026-02-22T21:54:35.955267+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=22, tm_hour=21, tm_min=54, tm_sec=35, tm_wday=6, tm_yday=53, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=22, tm_hour=21, tm_min=55, tm_sec=12, tm_wday=6, tm_yday=53, tm_isdst=0) 2026-02-22T21:55:14.388 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2026-02-22T21:55:34.389 DEBUG:teuthology.orchestra.run.trial093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T21:55:34.533 INFO:teuthology.orchestra.run.trial093.stdout: 2026-02-22T21:55:34.534 INFO:teuthology.orchestra.run.trial093.stderr:dumped all 2026-02-22T21:55:34.543 INFO:teuthology.orchestra.run.trial093.stdout:{"pg_ready":true,"pg_map":{"version":43,"stamp":"2026-02-22T21:55:32.954219+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":451,"num_read_kb":380,"num_write":116,"num_write_kb":1117,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":110,"ondisk_log_size":110,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":2,"num_per_pool_osds":2,"num_per_pool_omap_osds":2,"kb":188743680,"kb_used":55344,"kb_used_data":1424,"kb_used_omap":3,"kb_used_meta":53884,"kb_avail":188688336,"statfs":{"total":193273528320,"available":193216856064,"internally_reserved":0,"allocated":1458176,"data_stored":1251768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":3188,"internal_metadata":55178124},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001088"},"pg_stats":[{"pgid":"2.7","version":"15'31","reported_seq":193,"reported_epoch":15,"state":"active+clean","last_fresh":"2026-02-22T21:55:19.317914+0000","last_change":"2026-02-22T21:55:19.317946+0000","last_active":"2026-02-22T21:55:19.317914+0000","last_peered":"2026-02-22T21:55:19.317914+0000","last_clean":"2026-02-22T21:55:19.317914+0000","last_became_active":"2026-02-22T21:54:41.977824+0000","last_became_peered":"2026-02-22T21:54:41.977824+0000","last_unstale":"2026-02-22T21:55:19.317914+0000","last_undegraded":"2026-02-22T21:55:19.317914+0000","last_fullsized":"2026-02-22T21:55:19.317914+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"15'31","last_scrub_stamp":"2026-02-22T21:55:19.317829+0000","last_deep_scrub":"15'31","last_deep_scrub_stamp":"2026-02-22T21:55:19.317829+0000","last_clean_scrub_stamp":"2026-02-22T21:55:19.317829+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:31:07.860345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00078258400000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":319,"num_read_kb":283,"num_write":14,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"15'1","reported_seq":31,"reported_epoch":15,"state":"active+clean","last_fresh":"2026-02-22T21:55:18.365823+0000","last_change":"2026-02-22T21:55:18.365892+0000","last_active":"2026-02-22T21:55:18.365823+0000","last_peered":"2026-02-22T21:55:18.365823+0000","last_clean":"2026-02-22T21:55:18.365823+0000","last_became_active":"2026-02-22T21:54:41.977649+0000","last_became_peered":"2026-02-22T21:54:41.977649+0000","last_unstale":"2026-02-22T21:55:18.365823+0000","last_undegraded":"2026-02-22T21:55:18.365823+0000","last_fullsized":"2026-02-22T21:55:18.365823+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"15'1","last_scrub_stamp":"2026-02-22T21:55:18.365730+0000","last_deep_scrub":"15'1","last_deep_scrub_stamp":"2026-02-22T21:55:18.365730+0000","last_clean_scrub_stamp":"2026-02-22T21:55:18.365730+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:08:45.251014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00078215399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":30,"reported_epoch":15,"state":"active+clean","last_fresh":"2026-02-22T21:55:17.384579+0000","last_change":"2026-02-22T21:55:17.384607+0000","last_active":"2026-02-22T21:55:17.384579+0000","last_peered":"2026-02-22T21:55:17.384579+0000","last_clean":"2026-02-22T21:55:17.384579+0000","last_became_active":"2026-02-22T21:54:41.977655+0000","last_became_peered":"2026-02-22T21:54:41.977655+0000","last_unstale":"2026-02-22T21:55:17.384579+0000","last_undegraded":"2026-02-22T21:55:17.384579+0000","last_fullsized":"2026-02-22T21:55:17.384579+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T21:55:17.384501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T21:55:17.384501+0000","last_clean_scrub_stamp":"2026-02-22T21:55:17.384501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T01:37:26.247274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00038385300000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"15'4","reported_seq":54,"reported_epoch":15,"state":"active+clean","last_fresh":"2026-02-22T21:55:16.363283+0000","last_change":"2026-02-22T21:55:16.363312+0000","last_active":"2026-02-22T21:55:16.363283+0000","last_peered":"2026-02-22T21:55:16.363283+0000","last_clean":"2026-02-22T21:55:16.363283+0000","last_became_active":"2026-02-22T21:54:41.977538+0000","last_became_peered":"2026-02-22T21:54:41.977538+0000","last_unstale":"2026-02-22T21:55:16.363283+0000","last_undegraded":"2026-02-22T21:55:16.363283+0000","last_fullsized":"2026-02-22T21:55:16.363283+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"15'4","last_scrub_stamp":"2026-02-22T21:55:16.363173+0000","last_deep_scrub":"15'4","last_deep_scrub_stamp":"2026-02-22T21:55:16.363173+0000","last_clean_scrub_stamp":"2026-02-22T21:55:16.363173+0000","objects_scrubbed":1,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T00:24:11.745047+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0.00057796400000000004,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":33,"num_read_kb":15,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"15'3","reported_seq":49,"reported_epoch":15,"state":"active+clean","last_fresh":"2026-02-22T21:55:14.704346+0000","last_change":"2026-02-22T21:55:14.704366+0000","last_active":"2026-02-22T21:55:14.704346+0000","last_peered":"2026-02-22T21:55:14.704346+0000","last_clean":"2026-02-22T21:55:14.704346+0000","last_became_active":"2026-02-22T21:54:41.976693+0000","last_became_peered":"2026-02-22T21:54:41.976693+0000","last_unstale":"2026-02-22T21:55:14.704346+0000","last_undegraded":"2026-02-22T21:55:14.704346+0000","last_fullsized":"2026-02-22T21:55:14.704346+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"15'3","last_scrub_stamp":"2026-02-22T21:55:14.704295+0000","last_deep_scrub":"15'3","last_deep_scrub_stamp":"2026-02-22T21:55:14.704295+0000","last_clean_scrub_stamp":"2026-02-22T21:55:14.704295+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:29:44.678182+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0.00057252400000000004,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":16,"num_read_kb":16,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"15'12","reported_seq":47,"reported_epoch":15,"state":"active+clean","last_fresh":"2026-02-22T21:55:15.409637+0000","last_change":"2026-02-22T21:55:15.409663+0000","last_active":"2026-02-22T21:55:15.409637+0000","last_peered":"2026-02-22T21:55:15.409637+0000","last_clean":"2026-02-22T21:55:15.409637+0000","last_became_active":"2026-02-22T21:54:41.977454+0000","last_became_peered":"2026-02-22T21:54:41.977454+0000","last_unstale":"2026-02-22T21:55:15.409637+0000","last_undegraded":"2026-02-22T21:55:15.409637+0000","last_fullsized":"2026-02-22T21:55:15.409637+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"15'12","last_scrub_stamp":"2026-02-22T21:55:15.409566+0000","last_deep_scrub":"15'12","last_deep_scrub_stamp":"2026-02-22T21:55:15.409566+0000","last_clean_scrub_stamp":"2026-02-22T21:55:15.409566+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T09:18:00.935706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000616426,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":21,"num_read_kb":14,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":30,"reported_epoch":15,"state":"active+clean","last_fresh":"2026-02-22T21:55:15.722413+0000","last_change":"2026-02-22T21:55:15.722428+0000","last_active":"2026-02-22T21:55:15.722413+0000","last_peered":"2026-02-22T21:55:15.722413+0000","last_clean":"2026-02-22T21:55:15.722413+0000","last_became_active":"2026-02-22T21:54:41.977088+0000","last_became_peered":"2026-02-22T21:54:41.977088+0000","last_unstale":"2026-02-22T21:55:15.722413+0000","last_undegraded":"2026-02-22T21:55:15.722413+0000","last_fullsized":"2026-02-22T21:55:15.722413+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T21:55:15.722358+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T21:55:15.722358+0000","last_clean_scrub_stamp":"2026-02-22T21:55:15.722358+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T22:22:28.775078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00028617200000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.3","version":"15'5","reported_seq":39,"reported_epoch":15,"state":"active+clean","last_fresh":"2026-02-22T21:55:20.292856+0000","last_change":"2026-02-22T21:55:20.292905+0000","last_active":"2026-02-22T21:55:20.292856+0000","last_peered":"2026-02-22T21:55:20.292856+0000","last_clean":"2026-02-22T21:55:20.292856+0000","last_became_active":"2026-02-22T21:54:41.977469+0000","last_became_peered":"2026-02-22T21:54:41.977469+0000","last_unstale":"2026-02-22T21:55:20.292856+0000","last_undegraded":"2026-02-22T21:55:20.292856+0000","last_fullsized":"2026-02-22T21:55:20.292856+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"15'5","last_scrub_stamp":"2026-02-22T21:55:20.292776+0000","last_deep_scrub":"15'5","last_deep_scrub_stamp":"2026-02-22T21:55:20.292776+0000","last_clean_scrub_stamp":"2026-02-22T21:55:20.292776+0000","objects_scrubbed":1,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T02:48:38.517648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00039984299999999999,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":4,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"15'54","reported_seq":100,"reported_epoch":15,"state":"active+clean","last_fresh":"2026-02-22T21:55:14.415807+0000","last_change":"2026-02-22T21:55:14.415834+0000","last_active":"2026-02-22T21:55:14.415807+0000","last_peered":"2026-02-22T21:55:14.415807+0000","last_clean":"2026-02-22T21:55:14.415807+0000","last_became_active":"2026-02-22T21:54:37.358358+0000","last_became_peered":"2026-02-22T21:54:37.358358+0000","last_unstale":"2026-02-22T21:55:14.415807+0000","last_undegraded":"2026-02-22T21:55:14.415807+0000","last_fullsized":"2026-02-22T21:55:14.415807+0000","mapping_epoch":8,"log_start":"0'0","ondisk_log_start":"0'0","created":8,"last_epoch_clean":9,"parent":"0.0","parent_split_bits":0,"last_scrub":"15'54","last_scrub_stamp":"2026-02-22T21:55:14.415737+0000","last_deep_scrub":"15'54","last_deep_scrub_stamp":"2026-02-22T21:55:14.415737+0000","last_clean_scrub_stamp":"2026-02-22T21:55:14.415737+0000","objects_scrubbed":2,"log_size":54,"log_dups_size":0,"ondisk_log_size":54,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T08:17:49.994528+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":56,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":395,"num_read_kb":334,"num_write":37,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":56,"ondisk_log_size":56,"up":16,"acting":16,"num_store_stats":2},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":56,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":54,"ondisk_log_size":54,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":1,"up_from":7,"seq":30064771087,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27672,"kb_used_data":712,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344168,"statfs":{"total":96636764160,"available":96608428032,"internally_reserved":0,"allocated":729088,"data_stored":625884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27589062},"hb_peers":[0],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":7,"seq":30064771086,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27672,"kb_used_data":712,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344168,"statfs":{"total":96636764160,"available":96608428032,"internally_reserved":0,"allocated":729088,"data_stored":625884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27589062},"hb_peers":[1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T21:55:34.545 DEBUG:teuthology.orchestra.run.trial093:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2026-02-22T21:55:34.707 INFO:teuthology.misc:Shutting down mds daemons... 2026-02-22T21:55:34.707 INFO:teuthology.misc:Shutting down osd daemons... 2026-02-22T21:55:34.707 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2026-02-22T21:55:34.708 INFO:teuthology.orchestra.run:waiting for 300 2026-02-22T21:55:34.795 INFO:tasks.ceph.osd.0:Stopped 2026-02-22T21:55:34.795 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2026-02-22T21:55:34.795 INFO:teuthology.orchestra.run:waiting for 300 2026-02-22T21:55:34.884 INFO:tasks.ceph.osd.1:Stopped 2026-02-22T21:55:34.884 INFO:teuthology.misc:Shutting down mgr daemons... 2026-02-22T21:55:34.884 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2026-02-22T21:55:34.884 INFO:teuthology.orchestra.run:waiting for 300 2026-02-22T21:55:34.911 INFO:tasks.ceph.mgr.x:Stopped 2026-02-22T21:55:34.911 INFO:teuthology.misc:Shutting down mon daemons... 2026-02-22T21:55:34.911 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2026-02-22T21:55:34.911 INFO:teuthology.orchestra.run:waiting for 300 2026-02-22T21:55:34.963 INFO:tasks.ceph.mon.a:Stopped 2026-02-22T21:55:34.964 INFO:tasks.ceph:Checking cluster log for badness... 2026-02-22T21:55:34.964 DEBUG:teuthology.orchestra.run.trial093:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | head -n 1 2026-02-22T21:55:35.018 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@trial093.front.sepia.ceph.com 2026-02-22T21:55:35.018 DEBUG:teuthology.orchestra.run.trial093:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2026-02-22T21:55:35.073 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@trial093.front.sepia.ceph.com 2026-02-22T21:55:35.074 DEBUG:teuthology.orchestra.run.trial093:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2026-02-22T21:55:35.125 INFO:tasks.ceph:Archiving mon data... 2026-02-22T21:55:35.127 DEBUG:teuthology.misc:Transferring archived files from trial093:/var/lib/ceph/mon/ceph-a to /home/teuthworker/mnt/teuthology/yuriw-2026-02-22_19:35:38-rbd-wip-yuri6-testing-2026-02-22-1629-squid-distro-default-trial/64132/data/mon.a.tgz 2026-02-22T21:55:35.127 DEBUG:teuthology.orchestra.run.trial093:> mktemp 2026-02-22T21:55:35.171 INFO:teuthology.orchestra.run.trial093.stdout:/tmp/tmp.Y39Aj32A4O 2026-02-22T21:55:35.171 DEBUG:teuthology.orchestra.run.trial093:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.Y39Aj32A4O 2026-02-22T21:55:35.238 DEBUG:teuthology.orchestra.run.trial093:> sudo chmod 0666 /tmp/tmp.Y39Aj32A4O 2026-02-22T21:55:35.336 DEBUG:teuthology.orchestra.remote:trial093:/tmp/tmp.Y39Aj32A4O is 144KB 2026-02-22T21:55:35.346 DEBUG:teuthology.orchestra.run.trial093:> rm -fr /tmp/tmp.Y39Aj32A4O 2026-02-22T21:55:35.349 INFO:tasks.ceph:Cleaning ceph cluster... 2026-02-22T21:55:35.350 DEBUG:teuthology.orchestra.run.trial093:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2026-02-22T21:55:35.447 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2026-02-22T21:55:35.447 INFO:tasks.ceph:Archiving crash dumps... 2026-02-22T21:55:35.449 DEBUG:teuthology.misc:Transferring archived files from trial093:/var/lib/ceph/crash to /home/teuthworker/mnt/teuthology/yuriw-2026-02-22_19:35:38-rbd-wip-yuri6-testing-2026-02-22-1629-squid-distro-default-trial/64132/remote/trial093/crash 2026-02-22T21:55:35.450 DEBUG:teuthology.orchestra.run.trial093:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2026-02-22T21:55:35.495 INFO:tasks.ceph:Compressing logs... 2026-02-22T21:55:35.496 DEBUG:teuthology.orchestra.run.trial093:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2026-02-22T21:55:35.547 INFO:teuthology.orchestra.run.trial093.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.17496.log 2026-02-22T21:55:35.549 INFO:teuthology.orchestra.run.trial093.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.15014.log 2026-02-22T21:55:35.549 INFO:teuthology.orchestra.run.trial093.stderr:/var/log/ceph/ceph-client.admin.17496.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.17496.log.gz 2026-02-22T21:55:35.550 INFO:teuthology.orchestra.run.trial093.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.17691.log 2026-02-22T21:55:35.550 INFO:teuthology.orchestra.run.trial093.stderr:/var/log/ceph/ceph-client.admin.15014.log: gzip -5 0.0% --verbose -- replaced with /var/log/ceph/ceph-client.admin.15014.log.gz -- 2026-02-22T21:55:35.550 INFO:teuthology.orchestra.run.trial093.stderr: /var/log/ceph/ceph-client.admin.15093.log 2026-02-22T21:55:35.550 INFO:teuthology.orchestra.run.trial093.stderr:/var/log/ceph/ceph-client.admin.17691.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.17749.log 2026-02-22T21:55:35.550 INFO:teuthology.orchestra.run.trial093.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.17691.log.gz 2026-02-22T21:55:35.550 INFO:teuthology.orchestra.run.trial093.stderr:/var/log/ceph/ceph-client.admin.15093.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.17723.log 2026-02-22T21:55:35.550 INFO:teuthology.orchestra.run.trial093.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.15093.log.gz 2026-02-22T21:55:35.550 INFO:teuthology.orchestra.run.trial093.stderr:/var/log/ceph/ceph-client.admin.17749.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.14899.log 2026-02-22T21:55:35.550 INFO:teuthology.orchestra.run.trial093.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.17749.log.gz 2026-02-22T21:55:35.550 INFO:teuthology.orchestra.run.trial093.stderr:/var/log/ceph/ceph-client.admin.17723.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.17723.log.gz 2026-02-22T21:55:35.550 INFO:teuthology.orchestra.run.trial093.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.17621.log 2026-02-22T21:55:35.559 INFO:teuthology.orchestra.run.trial093.stderr:/var/log/ceph/ceph-client.admin.14899.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.14899.log.gz 2026-02-22T21:55:35.559 INFO:teuthology.orchestra.run.trial093.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.17212.log 2026-02-22T21:55:35.559 INFO:teuthology.orchestra.run.trial093.stderr:/var/log/ceph/ceph-client.admin.17621.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.17621.log.gz 2026-02-22T21:55:35.559 INFO:teuthology.orchestra.run.trial093.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.17853.log 2026-02-22T21:55:35.559 INFO:teuthology.orchestra.run.trial093.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.17059.log 2026-02-22T21:55:35.560 INFO:teuthology.orchestra.run.trial093.stderr:/var/log/ceph/ceph-client.admin.17212.log: /var/log/ceph/ceph-client.admin.17853.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.17212.log.gz 2026-02-22T21:55:35.560 INFO:teuthology.orchestra.run.trial093.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.17853.log.gz 2026-02-22T21:55:35.560 INFO:teuthology.orchestra.run.trial093.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.16958.log 2026-02-22T21:55:35.560 INFO:teuthology.orchestra.run.trial093.stderr:/var/log/ceph/ceph-client.admin.17059.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2026-02-22T21:55:35.560 INFO:teuthology.orchestra.run.trial093.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.17059.log.gz 2026-02-22T21:55:35.560 INFO:teuthology.orchestra.run.trial093.stderr:/var/log/ceph/ceph-client.admin.16958.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.17441.log 2026-02-22T21:55:35.560 INFO:teuthology.orchestra.run.trial093.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.16958.log.gz 2026-02-22T21:55:35.560 INFO:teuthology.orchestra.run.trial093.stderr:/var/log/ceph/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.17981.log 2026-02-22T21:55:35.560 INFO:teuthology.orchestra.run.trial093.stderr:/var/log/ceph/ceph-client.admin.17441.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.17441.log.gz 2026-02-22T21:55:35.560 INFO:teuthology.orchestra.run.trial093.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.17160.log 2026-02-22T21:55:35.560 INFO:teuthology.orchestra.run.trial093.stderr:/var/log/ceph/ceph-client.admin.17981.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2026-02-22T21:55:35.560 INFO:teuthology.orchestra.run.trial093.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.17981.log.gz 2026-02-22T21:55:35.561 INFO:teuthology.orchestra.run.trial093.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.17827.log 2026-02-22T21:55:35.561 INFO:teuthology.orchestra.run.trial093.stderr:/var/log/ceph/ceph-client.admin.17160.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.17160.log.gz 2026-02-22T21:55:35.561 INFO:teuthology.orchestra.run.trial093.stderr:/var/log/ceph/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.17111.log 2026-02-22T21:55:35.561 INFO:teuthology.orchestra.run.trial093.stderr:/var/log/ceph/ceph-client.admin.17827.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.17827.log.gz 2026-02-22T21:55:35.561 INFO:teuthology.orchestra.run.trial093.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.17353.log 2026-02-22T21:55:35.561 INFO:teuthology.orchestra.run.trial093.stderr:/var/log/ceph/ceph-client.admin.17111.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.17801.log 2026-02-22T21:55:35.561 INFO:teuthology.orchestra.run.trial093.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.17111.log.gz 2026-02-22T21:55:35.561 INFO:teuthology.orchestra.run.trial093.stderr:/var/log/ceph/ceph-client.admin.17353.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.17030.log 2026-02-22T21:55:35.561 INFO:teuthology.orchestra.run.trial093.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.17353.log.gz 2026-02-22T21:55:35.561 INFO:teuthology.orchestra.run.trial093.stderr:/var/log/ceph/ceph-client.admin.17801.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.17801.log.gz 2026-02-22T21:55:35.561 INFO:teuthology.orchestra.run.trial093.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.16978.log 2026-02-22T21:55:35.561 INFO:teuthology.orchestra.run.trial093.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.17164.log 2026-02-22T21:55:35.561 INFO:teuthology.orchestra.run.trial093.stderr:/var/log/ceph/ceph-client.admin.17030.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.17030.log.gz 2026-02-22T21:55:35.562 INFO:teuthology.orchestra.run.trial093.stderr:/var/log/ceph/ceph-client.admin.16978.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.17376.log 2026-02-22T21:55:35.562 INFO:teuthology.orchestra.run.trial093.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.16978.log.gz 2026-02-22T21:55:35.562 INFO:teuthology.orchestra.run.trial093.stderr:/var/log/ceph/ceph-client.admin.17164.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.17164.log.gz 2026-02-22T21:55:35.562 INFO:teuthology.orchestra.run.trial093.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.17592.log 2026-02-22T21:55:35.562 INFO:teuthology.orchestra.run.trial093.stderr:/var/log/ceph/ceph-client.admin.17376.log: 85.8% -- replaced with /var/log/ceph/ceph-client.admin.17376.log.gz 2026-02-22T21:55:35.562 INFO:teuthology.orchestra.run.trial093.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.17775.log 2026-02-22T21:55:35.562 INFO:teuthology.orchestra.run.trial093.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.17467.log 2026-02-22T21:55:35.562 INFO:teuthology.orchestra.run.trial093.stderr:/var/log/ceph/ceph-client.admin.17592.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.17592.log.gz 2026-02-22T21:55:35.562 INFO:teuthology.orchestra.run.trial093.stderr:/var/log/ceph/ceph-client.admin.17775.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.14990.log 2026-02-22T21:55:35.562 INFO:teuthology.orchestra.run.trial093.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.17775.log.gz 2026-02-22T21:55:35.562 INFO:teuthology.orchestra.run.trial093.stderr:/var/log/ceph/ceph-client.admin.17467.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.17467.log.gz 2026-02-22T21:55:35.562 INFO:teuthology.orchestra.run.trial093.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.17238.log 2026-02-22T21:55:35.562 INFO:teuthology.orchestra.run.trial093.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2026-02-22T21:55:35.563 INFO:teuthology.orchestra.run.trial093.stderr:/var/log/ceph/ceph-client.admin.14990.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.14990.log.gz 2026-02-22T21:55:35.563 INFO:teuthology.orchestra.run.trial093.stderr:/var/log/ceph/ceph-client.admin.17238.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.17238.log.gz 2026-02-22T21:55:35.563 INFO:teuthology.orchestra.run.trial093.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.17264.log 2026-02-22T21:55:35.563 INFO:teuthology.orchestra.run.trial093.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.15046.log 2026-02-22T21:55:35.563 INFO:teuthology.orchestra.run.trial093.stderr: 86.4% -- replaced with /var/log/ceph/ceph.audit.log.gz 2026-02-22T21:55:35.563 INFO:teuthology.orchestra.run.trial093.stderr:/var/log/ceph/ceph-client.admin.17264.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.17957.log 2026-02-22T21:55:35.563 INFO:teuthology.orchestra.run.trial093.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.17264.log.gz 2026-02-22T21:55:35.563 INFO:teuthology.orchestra.run.trial093.stderr:/var/log/ceph/ceph-client.admin.15046.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.15046.log.gz 2026-02-22T21:55:35.563 INFO:teuthology.orchestra.run.trial093.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.16933.log 2026-02-22T21:55:35.563 INFO:teuthology.orchestra.run.trial093.stderr: 91.3% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2026-02-22T21:55:35.563 INFO:teuthology.orchestra.run.trial093.stderr:/var/log/ceph/ceph-client.admin.17957.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.17957.log.gz 2026-02-22T21:55:35.563 INFO:teuthology.orchestra.run.trial093.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2026-02-22T21:55:35.568 INFO:teuthology.orchestra.run.trial093.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.17060.log 2026-02-22T21:55:35.568 INFO:teuthology.orchestra.run.trial093.stderr:/var/log/ceph/ceph-client.admin.16933.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.16933.log.gz 2026-02-22T21:55:35.568 INFO:teuthology.orchestra.run.trial093.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.17405.log 2026-02-22T21:55:35.568 INFO:teuthology.orchestra.run.trial093.stderr: 83.4% -- replaced with /var/log/ceph/ceph.log.gz 2026-02-22T21:55:35.568 INFO:teuthology.orchestra.run.trial093.stderr:/var/log/ceph/ceph-client.admin.17060.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.17931.log 2026-02-22T21:55:35.568 INFO:teuthology.orchestra.run.trial093.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.17060.log.gz 2026-02-22T21:55:35.569 INFO:teuthology.orchestra.run.trial093.stderr:/var/log/ceph/ceph-client.admin.17405.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2026-02-22T21:55:35.569 INFO:teuthology.orchestra.run.trial093.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.17405.log.gz 2026-02-22T21:55:35.569 INFO:teuthology.orchestra.run.trial093.stderr:/var/log/ceph/ceph-client.admin.17931.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.17112.log 2026-02-22T21:55:35.569 INFO:teuthology.orchestra.run.trial093.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.17931.log.gz 2026-02-22T21:55:35.569 INFO:teuthology.orchestra.run.trial093.stderr:/var/log/ceph/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.17566.log 2026-02-22T21:55:35.569 INFO:teuthology.orchestra.run.trial093.stderr:/var/log/ceph/ceph-client.admin.17112.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.16908.log 2026-02-22T21:55:35.569 INFO:teuthology.orchestra.run.trial093.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.17112.log.gz 2026-02-22T21:55:35.569 INFO:teuthology.orchestra.run.trial093.stderr:gzip/var/log/ceph/ceph-client.admin.17566.log: -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2026-02-22T21:55:35.569 INFO:teuthology.orchestra.run.trial093.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.17566.log.gz 2026-02-22T21:55:35.569 INFO:teuthology.orchestra.run.trial093.stderr:/var/log/ceph/ceph-client.admin.16908.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.16908.log.gz 2026-02-22T21:55:35.569 INFO:teuthology.orchestra.run.trial093.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.16853.log 2026-02-22T21:55:35.569 INFO:teuthology.orchestra.run.trial093.stderr:/var/log/ceph/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.17004.log 2026-02-22T21:55:35.569 INFO:teuthology.orchestra.run.trial093.stderr:/var/log/ceph/ceph-client.admin.16853.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.17879.log 2026-02-22T21:55:35.570 INFO:teuthology.orchestra.run.trial093.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.16853.log.gz 2026-02-22T21:55:35.570 INFO:teuthology.orchestra.run.trial093.stderr:/var/log/ceph/ceph-client.admin.17004.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.17905.log 2026-02-22T21:55:35.570 INFO:teuthology.orchestra.run.trial093.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.17004.log.gz 2026-02-22T21:55:35.570 INFO:teuthology.orchestra.run.trial093.stderr:/var/log/ceph/ceph-client.admin.17879.log: gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.12482.log 2026-02-22T21:55:35.570 INFO:teuthology.orchestra.run.trial093.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.17879.log.gz 2026-02-22T21:55:35.570 INFO:teuthology.orchestra.run.trial093.stderr:/var/log/ceph/ceph-client.admin.17905.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.17905.log.gz 2026-02-22T21:55:35.570 INFO:teuthology.orchestra.run.trial093.stderr:/var/log/ceph/ceph.tmp-client.admin.12482.log: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.12482.log.gz 2026-02-22T21:55:35.572 INFO:teuthology.orchestra.run.trial093.stderr: 91.2% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2026-02-22T21:55:35.591 INFO:teuthology.orchestra.run.trial093.stderr: 94.1% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2026-02-22T21:55:35.604 INFO:teuthology.orchestra.run.trial093.stderr: 94.4% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2026-02-22T21:55:35.605 INFO:teuthology.orchestra.run.trial093.stderr: 2026-02-22T21:55:35.605 INFO:teuthology.orchestra.run.trial093.stderr:real 0m0.064s 2026-02-22T21:55:35.605 INFO:teuthology.orchestra.run.trial093.stderr:user 0m0.126s 2026-02-22T21:55:35.605 INFO:teuthology.orchestra.run.trial093.stderr:sys 0m0.026s 2026-02-22T21:55:35.606 INFO:tasks.ceph:Archiving logs... 2026-02-22T21:55:35.606 DEBUG:teuthology.misc:Transferring archived files from trial093:/var/log/ceph to /home/teuthworker/mnt/teuthology/yuriw-2026-02-22_19:35:38-rbd-wip-yuri6-testing-2026-02-22-1629-squid-distro-default-trial/64132/remote/trial093/log 2026-02-22T21:55:35.608 DEBUG:teuthology.orchestra.run.trial093:> sudo tar c -f - -C /var/log/ceph -- . 2026-02-22T21:55:35.707 DEBUG:teuthology.run_tasks:Unwinding manager install 2026-02-22T21:55:35.727 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2026-02-22T21:55:35.728 DEBUG:teuthology.orchestra.run.trial093:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2026-02-22T21:55:35.829 DEBUG:teuthology.run_tasks:Unwinding manager clock 2026-02-22T21:55:35.840 INFO:teuthology.task.clock:Checking final clock skew... 2026-02-22T21:55:35.840 DEBUG:teuthology.orchestra.run.trial093:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2026-02-22T21:55:35.892 INFO:teuthology.orchestra.run.trial093.stdout: remote refid st t when poll reach delay offset jitter 2026-02-22T21:55:35.892 INFO:teuthology.orchestra.run.trial093.stdout:============================================================================== 2026-02-22T21:55:35.892 INFO:teuthology.orchestra.run.trial093.stdout: 69-10-208-170.r 128.9.176.30 2 u 37 64 3 65.281 +1.170 0.108 2026-02-22T21:55:35.892 INFO:teuthology.orchestra.run.trial093.stdout: ntp11.kernfusio 237.17.204.95 2 u 32 64 3 30.240 +0.408 0.088 2026-02-22T21:55:35.892 INFO:teuthology.orchestra.run.trial093.stdout: us-east-2.clear 130.207.244.240 2 u 35 64 3 37.036 +1.100 0.123 2026-02-22T21:55:35.892 INFO:teuthology.orchestra.run.trial093.stdout: dns-e.ns4v.icu 228.188.15.147 2 u 37 64 3 62.003 -1.326 0.165 2026-02-22T21:55:35.893 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2026-02-22T21:55:35.904 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2026-02-22T21:55:35.904 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2026-02-22T21:55:35.918 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2026-02-22T21:55:35.934 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2026-02-22T21:55:35.948 INFO:teuthology.task.internal:Duration was 255.631251 seconds 2026-02-22T21:55:35.949 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2026-02-22T21:55:35.963 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2026-02-22T21:55:35.964 DEBUG:teuthology.orchestra.run.trial093:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2026-02-22T21:55:35.990 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2026-02-22T21:55:35.990 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@trial093.front.sepia.ceph.com 2026-02-22T21:55:35.990 DEBUG:teuthology.orchestra.run.trial093:> grep -E --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | grep -E -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | grep -E -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2026-02-22T21:55:36.037 INFO:teuthology.task.internal.syslog:Gathering journactl... 2026-02-22T21:55:36.038 DEBUG:teuthology.orchestra.run.trial093:> sudo journalctl > /home/ubuntu/cephtest/archive/syslog/journalctl.log 2026-02-22T21:55:36.294 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2026-02-22T21:55:36.294 DEBUG:teuthology.orchestra.run.trial093:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --max-args=1 --max-procs=0 --verbose --no-run-if-empty -- gzip -5 --verbose -- 2026-02-22T21:55:36.302 INFO:teuthology.orchestra.run.trial093.stderr:gzip -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/misc.log 2026-02-22T21:55:36.303 INFO:teuthology.orchestra.run.trial093.stderr:gzip -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/journalctl.log 2026-02-22T21:55:36.303 INFO:teuthology.orchestra.run.trial093.stderr:/home/ubuntu/cephtest/archive/syslog/misc.log: 0.0% -- replaced with /home/ubuntu/cephtest/archive/syslog/misc.log.gz 2026-02-22T21:55:36.303 INFO:teuthology.orchestra.run.trial093.stderr:gzip -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/kern.log 2026-02-22T21:55:36.303 INFO:teuthology.orchestra.run.trial093.stderr:/home/ubuntu/cephtest/archive/syslog/journalctl.log: /home/ubuntu/cephtest/archive/syslog/kern.log: 0.0% -- replaced with /home/ubuntu/cephtest/archive/syslog/kern.log.gz 2026-02-22T21:55:36.373 INFO:teuthology.orchestra.run.trial093.stderr: 81.4% -- replaced with /home/ubuntu/cephtest/archive/syslog/journalctl.log.gz 2026-02-22T21:55:36.375 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2026-02-22T21:55:36.386 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2026-02-22T21:55:36.386 DEBUG:teuthology.orchestra.run.trial093:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2026-02-22T21:55:36.424 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2026-02-22T21:55:36.437 DEBUG:teuthology.orchestra.run.trial093:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2026-02-22T21:55:36.479 INFO:teuthology.orchestra.run.trial093.stdout:kernel.core_pattern = core 2026-02-22T21:55:36.489 DEBUG:teuthology.orchestra.run.trial093:> test -e /home/ubuntu/cephtest/archive/coredump 2026-02-22T21:55:36.534 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-22T21:55:36.535 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2026-02-22T21:55:36.551 INFO:teuthology.task.internal:Transferring archived files... 2026-02-22T21:55:36.552 DEBUG:teuthology.misc:Transferring archived files from trial093:/home/ubuntu/cephtest/archive to /home/teuthworker/mnt/teuthology/yuriw-2026-02-22_19:35:38-rbd-wip-yuri6-testing-2026-02-22-1629-squid-distro-default-trial/64132/remote/trial093 2026-02-22T21:55:36.552 DEBUG:teuthology.orchestra.run.trial093:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2026-02-22T21:55:36.621 INFO:teuthology.task.internal:Removing archive directory... 2026-02-22T21:55:36.621 DEBUG:teuthology.orchestra.run.trial093:> rm -rf -- /home/ubuntu/cephtest/archive 2026-02-22T21:55:36.647 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2026-02-22T21:55:36.665 INFO:teuthology.task.internal:Not uploading archives. 2026-02-22T21:55:36.665 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2026-02-22T21:55:36.685 INFO:teuthology.task.internal:Tidying up after the test... 2026-02-22T21:55:36.685 DEBUG:teuthology.orchestra.run.trial093:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2026-02-22T21:55:36.699 INFO:teuthology.orchestra.run.trial093.stdout: 1179671 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Feb 22 21:55 /home/ubuntu/cephtest 2026-02-22T21:55:36.701 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2026-02-22T21:55:36.716 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2026-02-22T21:55:36.742 INFO:teuthology.run:Summary data: description: rbd/singleton/{all/admin_socket conf/{disable-pool-app} objectstore/bluestore-hybrid openstack supported-random-distro$/{ubuntu_latest}} duration: 255.6312506198883 flavor: default owner: scheduled_yuriw@soko04.front.sepia.ceph.com success: true 2026-02-22T21:55:36.742 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com 2026-02-22T21:55:36.851 INFO:teuthology.run:pass