2026-02-22T22:55:53.051 INFO:root:teuthology version: 1.2.4.dev9+gfa17720d0 2026-02-22T22:55:53.066 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com 2026-02-22T22:55:53.146 INFO:teuthology.run:Config: archive_path: /home/teuthworker/mnt/teuthology/yuriw-2026-02-22_19:35:38-rbd-wip-yuri6-testing-2026-02-22-1629-squid-distro-default-trial/64300 branch: wip-yuri6-testing-2026-02-22-1629-squid description: rbd/valgrind/{base/install centos_latest clusters/{fixed-1 openstack} conf/{disable-pool-app} objectstore/bluestore-hybrid validator/memcheck workloads/python_api_tests} email: yweinste@redhat.com first_in_suite: false flavor: default job_id: '64300' kernel: branch: distro kdb: true sha1: distro last_in_suite: false machine_type: trial name: yuriw-2026-02-22_19:35:38-rbd-wip-yuri6-testing-2026-02-22-1629-squid-distro-default-trial no_nested_subset: false openstack: - volumes: count: 3 size: 30 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-yuri6-testing-2026-02-22-1629-squid ceph: conf: global: mon warn on pool no app: false mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 osd: bluefs allocator: hybrid bluestore allocator: hybrid bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd shutdown pgref assert: true flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) sha1: f90f4eaebb28b619cf76e80cbbe997656da69579 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log global: osd crush chooseleaf type: 0 osd pool default pg num: 128 osd pool default pgp num: 128 osd pool default size: 2 mon: {} osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: debuginfo: true flavor: default sha1: f90f4eaebb28b619cf76e80cbbe997656da69579 extra_system_packages: - python3-pytest rbd_fsx: valgrind: - --tool=memcheck selinux: allowlist: - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-yuri6-testing-2026-02-22-1629-squid env: VALGRIND: --tool=memcheck --leak-check=full sha1: f90f4eaebb28b619cf76e80cbbe997656da69579 owner: scheduled_yuriw@soko04.front.sepia.ceph.com priority: 69 repo: https://github.com/ceph/ceph-ci.git roles: - - mon.a - mgr.x - osd.0 - osd.1 - osd.2 - client.0 seed: 9196 sha1: f90f4eaebb28b619cf76e80cbbe997656da69579 sleep_before_teardown: 0 subset: 111/120000 suite: rbd suite_branch: wip-yuri6-testing-2026-02-22-1629-squid suite_path: /home/teuthworker/src/github.com_ceph_ceph-c_f90f4eaebb28b619cf76e80cbbe997656da69579/qa suite_relpath: qa suite_repo: https://github.com/ceph/ceph-ci.git suite_sha1: f90f4eaebb28b619cf76e80cbbe997656da69579 targets: trial168.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHNIO79a5opEL90QIdZsj0tbsZXOM31K1RPGlxWiNLr18RNdFHu6GFQWSFmgCQIkcUzEd0LsquwlIvlBwJNKbHU= tasks: - install: null - ceph: null - workunit: clients: client.0: - rbd/test_librbd_python.sh env: RBD_FEATURES: '1' teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: fa17720d0088c3ac28e473468bfc79eeaff5cd38 timestamp: 2026-02-22_19:35:38 tube: trial user: yuriw verbose: true worker_log: /home/teuthworker/mnt/teuthology/worker_logs/dispatcher.trial.3318653 2026-02-22T22:55:53.146 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_ceph_ceph-c_f90f4eaebb28b619cf76e80cbbe997656da69579/qa; will attempt to use it 2026-02-22T22:55:53.146 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_ceph_ceph-c_f90f4eaebb28b619cf76e80cbbe997656da69579/qa/tasks 2026-02-22T22:55:53.147 INFO:teuthology.run_tasks:Running task internal.check_packages... 2026-02-22T22:55:53.148 INFO:teuthology.task.internal:Checking packages... 2026-02-22T22:55:53.148 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash 'f90f4eaebb28b619cf76e80cbbe997656da69579' 2026-02-22T22:55:53.148 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2026-02-22T22:55:53.148 INFO:teuthology.packaging:ref: None 2026-02-22T22:55:53.148 INFO:teuthology.packaging:tag: None 2026-02-22T22:55:53.148 INFO:teuthology.packaging:branch: wip-yuri6-testing-2026-02-22-1629-squid 2026-02-22T22:55:53.149 INFO:teuthology.packaging:sha1: f90f4eaebb28b619cf76e80cbbe997656da69579 2026-02-22T22:55:53.149 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-yuri6-testing-2026-02-22-1629-squid 2026-02-22T22:55:53.268 INFO:teuthology.task.internal:Found packages for ceph version 19.2.3-689.gf90f4eae 2026-02-22T22:55:53.270 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2026-02-22T22:55:53.281 INFO:teuthology.task.internal:no buildpackages task found 2026-02-22T22:55:53.281 INFO:teuthology.run_tasks:Running task internal.save_config... 2026-02-22T22:55:53.293 INFO:teuthology.task.internal:Saving configuration 2026-02-22T22:55:53.303 INFO:teuthology.run_tasks:Running task internal.check_lock... 2026-02-22T22:55:53.314 INFO:teuthology.task.internal.check_lock:Checking locks... 2026-02-22T22:55:53.391 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'trial168.front.sepia.ceph.com', 'description': '/home/teuthworker/mnt/teuthology/yuriw-2026-02-22_19:35:38-rbd-wip-yuri6-testing-2026-02-22-1629-squid-distro-default-trial/64300', 'up': True, 'machine_type': 'trial', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2026-02-22 22:45:46.447575', 'locked_by': 'scheduled_yuriw@soko04.front.sepia.ceph.com', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHNIO79a5opEL90QIdZsj0tbsZXOM31K1RPGlxWiNLr18RNdFHu6GFQWSFmgCQIkcUzEd0LsquwlIvlBwJNKbHU='} 2026-02-22T22:55:53.391 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2026-02-22T22:55:53.414 INFO:teuthology.task.internal:roles: ubuntu@trial168.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'osd.0', 'osd.1', 'osd.2', 'client.0'] 2026-02-22T22:55:53.414 INFO:teuthology.run_tasks:Running task console_log... 2026-02-22T22:55:53.512 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7ba97cbf7a60>, signals=[15]) 2026-02-22T22:55:53.512 INFO:teuthology.run_tasks:Running task internal.connect... 2026-02-22T22:55:53.524 INFO:teuthology.task.internal:Opening connections... 2026-02-22T22:55:53.524 DEBUG:teuthology.task.internal:connecting to ubuntu@trial168.front.sepia.ceph.com 2026-02-22T22:55:53.525 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial168.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2026-02-22T22:55:53.589 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2026-02-22T22:55:53.601 DEBUG:teuthology.orchestra.run.trial168:> uname -m 2026-02-22T22:55:53.661 INFO:teuthology.orchestra.run.trial168.stdout:x86_64 2026-02-22T22:55:53.661 DEBUG:teuthology.orchestra.run.trial168:> cat /etc/os-release 2026-02-22T22:55:53.716 INFO:teuthology.orchestra.run.trial168.stdout:NAME="CentOS Stream" 2026-02-22T22:55:53.716 INFO:teuthology.orchestra.run.trial168.stdout:VERSION="9" 2026-02-22T22:55:53.716 INFO:teuthology.orchestra.run.trial168.stdout:ID="centos" 2026-02-22T22:55:53.716 INFO:teuthology.orchestra.run.trial168.stdout:ID_LIKE="rhel fedora" 2026-02-22T22:55:53.716 INFO:teuthology.orchestra.run.trial168.stdout:VERSION_ID="9" 2026-02-22T22:55:53.716 INFO:teuthology.orchestra.run.trial168.stdout:PLATFORM_ID="platform:el9" 2026-02-22T22:55:53.716 INFO:teuthology.orchestra.run.trial168.stdout:PRETTY_NAME="CentOS Stream 9" 2026-02-22T22:55:53.716 INFO:teuthology.orchestra.run.trial168.stdout:ANSI_COLOR="0;31" 2026-02-22T22:55:53.716 INFO:teuthology.orchestra.run.trial168.stdout:LOGO="fedora-logo-icon" 2026-02-22T22:55:53.716 INFO:teuthology.orchestra.run.trial168.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2026-02-22T22:55:53.716 INFO:teuthology.orchestra.run.trial168.stdout:HOME_URL="https://centos.org/" 2026-02-22T22:55:53.716 INFO:teuthology.orchestra.run.trial168.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2026-02-22T22:55:53.716 INFO:teuthology.orchestra.run.trial168.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2026-02-22T22:55:53.716 INFO:teuthology.orchestra.run.trial168.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2026-02-22T22:55:53.717 INFO:teuthology.lock.ops:Updating trial168.front.sepia.ceph.com on lock server 2026-02-22T22:55:53.802 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2026-02-22T22:55:53.814 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2026-02-22T22:55:53.827 INFO:teuthology.task.internal:Checking for old test directory... 2026-02-22T22:55:53.827 DEBUG:teuthology.orchestra.run.trial168:> test '!' -e /home/ubuntu/cephtest 2026-02-22T22:55:53.843 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2026-02-22T22:55:53.854 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2026-02-22T22:55:53.854 DEBUG:teuthology.orchestra.run.trial168:> test -z $(ls -A /var/lib/ceph) 2026-02-22T22:55:53.900 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2026-02-22T22:55:53.975 INFO:teuthology.run_tasks:Running task kernel... 2026-02-22T22:55:53.995 INFO:teuthology.task.kernel:normalize config orig: {'branch': 'distro', 'kdb': True, 'sha1': 'distro'} 2026-02-22T22:55:53.995 INFO:teuthology.task.kernel:config {'mon.a': {'branch': 'distro', 'kdb': True, 'sha1': 'distro'}}, timeout 300 2026-02-22T22:55:53.996 DEBUG:teuthology.orchestra.run.trial168:> test -f /run/.containerenv -o -f /.dockerenv 2026-02-22T22:55:54.012 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-22T22:55:54.012 DEBUG:teuthology.orchestra.run.trial168:> uname -r 2026-02-22T22:55:54.067 INFO:teuthology.orchestra.run.trial168.stdout:5.14.0-681.el9.x86_64 2026-02-22T22:55:54.067 INFO:teuthology.task.kernel:Running kernel on trial168: 5.14.0-681.el9.x86_64 2026-02-22T22:55:54.067 DEBUG:teuthology.orchestra.run.trial168:> sudo yum install -y kernel 2026-02-22T22:55:55.282 INFO:teuthology.orchestra.run.trial168.stdout:CentOS Stream 9 - BaseOS 11 MB/s | 8.9 MB 00:00 2026-02-22T22:55:58.433 INFO:teuthology.orchestra.run.trial168.stdout:CentOS Stream 9 - AppStream 10 MB/s | 27 MB 00:02 2026-02-22T22:56:01.822 INFO:teuthology.orchestra.run.trial168.stdout:CentOS Stream 9 - CRB 7.5 MB/s | 7.8 MB 00:01 2026-02-22T22:56:02.846 INFO:teuthology.orchestra.run.trial168.stdout:CentOS Stream 9 - Extras packages 66 kB/s | 20 kB 00:00 2026-02-22T22:56:03.913 INFO:teuthology.orchestra.run.trial168.stdout:Extra Packages for Enterprise Linux 20 MB/s | 20 MB 00:01 2026-02-22T22:56:07.082 INFO:teuthology.orchestra.run.trial168.stdout:lab-extras 1.7 MB/s | 50 kB 00:00 2026-02-22T22:56:08.098 INFO:teuthology.orchestra.run.trial168.stdout:Package kernel-5.14.0-645.el9.x86_64 is already installed. 2026-02-22T22:56:08.099 INFO:teuthology.orchestra.run.trial168.stdout:Package kernel-5.14.0-681.el9.x86_64 is already installed. 2026-02-22T22:56:08.117 INFO:teuthology.orchestra.run.trial168.stdout:Dependencies resolved. 2026-02-22T22:56:08.118 INFO:teuthology.orchestra.run.trial168.stdout:Nothing to do. 2026-02-22T22:56:08.118 INFO:teuthology.orchestra.run.trial168.stdout:Complete! 2026-02-22T22:56:08.143 DEBUG:teuthology.orchestra.run.trial168:> echo no | sudo yum reinstall kernel || true 2026-02-22T22:56:08.448 INFO:teuthology.orchestra.run.trial168.stdout:Last metadata expiration check: 0:00:01 ago on Sun 22 Feb 2026 10:56:07 PM UTC. 2026-02-22T22:56:08.529 INFO:teuthology.orchestra.run.trial168.stdout:Dependencies resolved. 2026-02-22T22:56:08.530 INFO:teuthology.orchestra.run.trial168.stdout:================================================================================ 2026-02-22T22:56:08.531 INFO:teuthology.orchestra.run.trial168.stdout: Package Architecture Version Repository Size 2026-02-22T22:56:08.531 INFO:teuthology.orchestra.run.trial168.stdout:================================================================================ 2026-02-22T22:56:08.531 INFO:teuthology.orchestra.run.trial168.stdout:Reinstalling: 2026-02-22T22:56:08.531 INFO:teuthology.orchestra.run.trial168.stdout: kernel x86_64 5.14.0-681.el9 baseos 803 k 2026-02-22T22:56:08.531 INFO:teuthology.orchestra.run.trial168.stdout: 2026-02-22T22:56:08.531 INFO:teuthology.orchestra.run.trial168.stdout:Transaction Summary 2026-02-22T22:56:08.531 INFO:teuthology.orchestra.run.trial168.stdout:================================================================================ 2026-02-22T22:56:08.531 INFO:teuthology.orchestra.run.trial168.stdout: 2026-02-22T22:56:08.531 INFO:teuthology.orchestra.run.trial168.stdout:Total download size: 803 k 2026-02-22T22:56:08.531 INFO:teuthology.orchestra.run.trial168.stdout:Installed size: 0 2026-02-22T22:56:08.532 INFO:teuthology.orchestra.run.trial168.stderr:Operation aborted. 2026-02-22T22:56:08.551 INFO:teuthology.orchestra.run.trial168.stdout:Is this ok [y/N]: 2026-02-22T22:56:08.552 DEBUG:teuthology.orchestra.run.trial168:> sudo yum reinstall -y kernel || true 2026-02-22T22:56:08.851 INFO:teuthology.orchestra.run.trial168.stdout:Last metadata expiration check: 0:00:01 ago on Sun 22 Feb 2026 10:56:07 PM UTC. 2026-02-22T22:56:08.931 INFO:teuthology.orchestra.run.trial168.stdout:Dependencies resolved. 2026-02-22T22:56:08.932 INFO:teuthology.orchestra.run.trial168.stdout:================================================================================ 2026-02-22T22:56:08.932 INFO:teuthology.orchestra.run.trial168.stdout: Package Architecture Version Repository Size 2026-02-22T22:56:08.932 INFO:teuthology.orchestra.run.trial168.stdout:================================================================================ 2026-02-22T22:56:08.932 INFO:teuthology.orchestra.run.trial168.stdout:Reinstalling: 2026-02-22T22:56:08.932 INFO:teuthology.orchestra.run.trial168.stdout: kernel x86_64 5.14.0-681.el9 baseos 803 k 2026-02-22T22:56:08.932 INFO:teuthology.orchestra.run.trial168.stdout: 2026-02-22T22:56:08.932 INFO:teuthology.orchestra.run.trial168.stdout:Transaction Summary 2026-02-22T22:56:08.933 INFO:teuthology.orchestra.run.trial168.stdout:================================================================================ 2026-02-22T22:56:08.933 INFO:teuthology.orchestra.run.trial168.stdout: 2026-02-22T22:56:08.933 INFO:teuthology.orchestra.run.trial168.stdout:Total download size: 803 k 2026-02-22T22:56:08.933 INFO:teuthology.orchestra.run.trial168.stdout:Installed size: 0 2026-02-22T22:56:08.933 INFO:teuthology.orchestra.run.trial168.stdout:Downloading Packages: 2026-02-22T22:56:10.378 INFO:teuthology.orchestra.run.trial168.stdout:kernel-5.14.0-681.el9.x86_64.rpm 579 kB/s | 803 kB 00:01 2026-02-22T22:56:10.379 INFO:teuthology.orchestra.run.trial168.stdout:-------------------------------------------------------------------------------- 2026-02-22T22:56:10.379 INFO:teuthology.orchestra.run.trial168.stdout:Total 555 kB/s | 803 kB 00:01 2026-02-22T22:56:10.392 INFO:teuthology.orchestra.run.trial168.stdout:Running transaction check 2026-02-22T22:56:10.429 INFO:teuthology.orchestra.run.trial168.stdout:Transaction check succeeded. 2026-02-22T22:56:10.429 INFO:teuthology.orchestra.run.trial168.stdout:Running transaction test 2026-02-22T22:56:10.433 INFO:teuthology.orchestra.run.trial168.stdout:Transaction test succeeded. 2026-02-22T22:56:10.434 INFO:teuthology.orchestra.run.trial168.stdout:Running transaction 2026-02-22T22:56:10.458 INFO:teuthology.orchestra.run.trial168.stdout: Preparing : 1/1 2026-02-22T22:56:10.461 INFO:teuthology.orchestra.run.trial168.stdout: Reinstalling : kernel-5.14.0-681.el9.x86_64 1/2 2026-02-22T22:56:10.462 INFO:teuthology.orchestra.run.trial168.stdout: Cleanup : kernel-5.14.0-681.el9.x86_64 2/2 2026-02-22T22:56:10.585 INFO:teuthology.orchestra.run.trial168.stdout: Running scriptlet: kernel-5.14.0-681.el9.x86_64 2/2 2026-02-22T22:56:10.585 INFO:teuthology.orchestra.run.trial168.stdout: Verifying : kernel-5.14.0-681.el9.x86_64 1/2 2026-02-22T22:56:10.641 INFO:teuthology.orchestra.run.trial168.stdout: Verifying : kernel-5.14.0-681.el9.x86_64 2/2 2026-02-22T22:56:10.641 INFO:teuthology.orchestra.run.trial168.stdout: 2026-02-22T22:56:10.641 INFO:teuthology.orchestra.run.trial168.stdout:Reinstalled: 2026-02-22T22:56:10.642 INFO:teuthology.orchestra.run.trial168.stdout: kernel-5.14.0-681.el9.x86_64 2026-02-22T22:56:10.642 INFO:teuthology.orchestra.run.trial168.stdout: 2026-02-22T22:56:10.642 INFO:teuthology.orchestra.run.trial168.stdout:Complete! 2026-02-22T22:56:10.670 DEBUG:teuthology.orchestra.run.trial168:> rpm -q kernel | sort -rV | head -n 1 2026-02-22T22:56:10.694 INFO:teuthology.orchestra.run.trial168.stdout:kernel-5.14.0-681.el9.x86_64 2026-02-22T22:56:10.694 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-681.el9.x86_64 2026-02-22T22:56:10.694 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2026-02-22T22:56:10.695 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2026-02-22T22:56:10.695 DEBUG:teuthology.orchestra.run.trial168:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2026-02-22T22:56:10.759 INFO:teuthology.orchestra.run.trial168.stdout:ttyS1 2026-02-22T22:56:10.780 DEBUG:teuthology.parallel:result is None 2026-02-22T22:56:10.780 INFO:teuthology.run_tasks:Running task internal.base... 2026-02-22T22:56:10.793 INFO:teuthology.task.internal:Creating test directory... 2026-02-22T22:56:10.794 DEBUG:teuthology.orchestra.run.trial168:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2026-02-22T22:56:10.837 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2026-02-22T22:56:10.850 INFO:teuthology.run_tasks:Running task internal.archive... 2026-02-22T22:56:10.862 INFO:teuthology.task.internal:Creating archive directory... 2026-02-22T22:56:10.862 DEBUG:teuthology.orchestra.run.trial168:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2026-02-22T22:56:10.901 INFO:teuthology.run_tasks:Running task internal.coredump... 2026-02-22T22:56:10.913 INFO:teuthology.task.internal:Enabling coredump saving... 2026-02-22T22:56:10.914 DEBUG:teuthology.orchestra.run.trial168:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2026-02-22T22:56:10.960 INFO:teuthology.orchestra.run.trial168.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-22T22:56:10.969 INFO:teuthology.orchestra.run.trial168.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-22T22:56:10.971 INFO:teuthology.run_tasks:Running task internal.sudo... 2026-02-22T22:56:10.983 INFO:teuthology.task.internal:Configuring sudo... 2026-02-22T22:56:10.983 DEBUG:teuthology.orchestra.run.trial168:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2026-02-22T22:56:11.036 INFO:teuthology.run_tasks:Running task internal.syslog... 2026-02-22T22:56:11.055 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2026-02-22T22:56:11.055 DEBUG:teuthology.orchestra.run.trial168:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2026-02-22T22:56:11.092 DEBUG:teuthology.orchestra.run.trial168:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2026-02-22T22:56:11.252 DEBUG:teuthology.orchestra.run.trial168:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2026-02-22T22:56:11.278 DEBUG:teuthology.orchestra.run.trial168:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2026-02-22T22:56:11.423 DEBUG:teuthology.orchestra.run.trial168:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2026-02-22T22:56:11.446 DEBUG:teuthology.orchestra.run.trial168:> set -ex 2026-02-22T22:56:11.446 DEBUG:teuthology.orchestra.run.trial168:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2026-02-22T22:56:11.511 DEBUG:teuthology.orchestra.run.trial168:> sudo service rsyslog restart 2026-02-22T22:56:11.579 INFO:teuthology.orchestra.run.trial168.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2026-02-22T22:56:12.101 INFO:teuthology.run_tasks:Running task internal.timer... 2026-02-22T22:56:12.114 INFO:teuthology.task.internal:Starting timer... 2026-02-22T22:56:12.114 INFO:teuthology.run_tasks:Running task pcp... 2026-02-22T22:56:12.137 INFO:teuthology.run_tasks:Running task selinux... 2026-02-22T22:56:12.155 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:getty_t:s0']} 2026-02-22T22:56:12.155 DEBUG:teuthology.orchestra.run.trial168:> sudo service auditd rotate 2026-02-22T22:56:12.185 INFO:teuthology.orchestra.run.trial168.stdout:Rotating logs: 2026-02-22T22:56:12.187 DEBUG:teuthology.task.selinux:Getting current SELinux state 2026-02-22T22:56:12.187 DEBUG:teuthology.orchestra.run.trial168:> /usr/sbin/getenforce 2026-02-22T22:56:12.245 INFO:teuthology.orchestra.run.trial168.stdout:Permissive 2026-02-22T22:56:12.246 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@trial168.front.sepia.ceph.com': 'permissive'} 2026-02-22T22:56:12.246 DEBUG:teuthology.orchestra.run.trial168:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2026-02-22T22:56:12.310 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-22T22:56:12.310 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2026-02-22T22:56:12.310 DEBUG:teuthology.orchestra.run.trial168:> sudo /usr/sbin/setenforce permissive 2026-02-22T22:56:12.373 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2026-02-22T22:56:12.389 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2026-02-22T22:56:12.391 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2026-02-22T22:56:12.501 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2026-02-22T22:56:12.514 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2026-02-22T22:56:12.515 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit trial168.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2026-02-22T22:58:56.177 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@trial168.front.sepia.ceph.com')] 2026-02-22T22:58:56.180 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@trial168.front.sepia.ceph.com' 2026-02-22T22:58:56.183 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial168.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2026-02-22T22:58:56.259 DEBUG:teuthology.orchestra.run.trial168:> true 2026-02-22T22:58:56.335 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@trial168.front.sepia.ceph.com' 2026-02-22T22:58:56.335 INFO:teuthology.run_tasks:Running task clock... 2026-02-22T22:58:56.349 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2026-02-22T22:58:56.349 INFO:teuthology.orchestra.run:Running command with timeout 360 2026-02-22T22:58:56.350 DEBUG:teuthology.orchestra.run.trial168:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2026-02-22T22:58:56.403 INFO:teuthology.orchestra.run.trial168.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2026-02-22T22:58:56.414 INFO:teuthology.orchestra.run.trial168.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2026-02-22T22:58:56.433 INFO:teuthology.orchestra.run.trial168.stderr:sudo: ntpd: command not found 2026-02-22T22:58:56.443 INFO:teuthology.orchestra.run.trial168.stdout:506 Cannot talk to daemon 2026-02-22T22:58:56.453 INFO:teuthology.orchestra.run.trial168.stderr:Failed to start ntp.service: Unit ntp.service not found. 2026-02-22T22:58:56.464 INFO:teuthology.orchestra.run.trial168.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2026-02-22T22:58:56.506 INFO:teuthology.orchestra.run.trial168.stderr:bash: line 1: ntpq: command not found 2026-02-22T22:58:56.508 INFO:teuthology.orchestra.run.trial168.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2026-02-22T22:58:56.508 INFO:teuthology.orchestra.run.trial168.stdout:=============================================================================== 2026-02-22T22:58:56.509 INFO:teuthology.run_tasks:Running task install... 2026-02-22T22:58:56.523 DEBUG:teuthology.task.install:project ceph 2026-02-22T22:58:56.523 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'debuginfo': True, 'flavor': 'default', 'sha1': 'f90f4eaebb28b619cf76e80cbbe997656da69579'}, 'extra_system_packages': ['python3-pytest']} 2026-02-22T22:58:56.523 DEBUG:teuthology.task.install:config {'debuginfo': True, 'flavor': 'default', 'sha1': 'f90f4eaebb28b619cf76e80cbbe997656da69579', 'extra_system_packages': {'deb': ['python3-pytest'], 'rpm': ['python3-pytest']}} 2026-02-22T22:58:56.524 INFO:teuthology.task.install:Using flavor: default 2026-02-22T22:58:56.533 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse', 'ceph-common-dbg', 'ceph-fuse-dbg', 'ceph-mds-dbg', 'ceph-mgr-dbg', 'ceph-mon-dbg', 'ceph-osd-dbg', 'ceph-test-dbg', 'libcephfs2-dbg', 'librados2-dbg', 'libradosstriper1-dbg', 'librbd1-dbg', 'librgw2-dbg', 'radosgw-dbg', 'rbd-fuse-dbg', 'rbd-mirror-dbg', 'rbd-nbd-dbg'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd', 'ceph-base-debuginfo', 'ceph-common-debuginfo', 'ceph-immutable-object-cache-debuginfo', 'ceph-radosgw-debuginfo', 'ceph-test-debuginfo', 'ceph-base-debuginfo', 'ceph-mgr-debuginfo', 'ceph-mds-debuginfo', 'ceph-mon-debuginfo', 'ceph-osd-debuginfo', 'ceph-fuse-debuginfo', 'librados-devel-debuginfo', 'libcephfs2-debuginfo', 'librados2-debuginfo', 'librbd1-debuginfo', 'python3-cephfs-debuginfo', 'python3-rados-debuginfo', 'python3-rbd-debuginfo', 'python3-rgw-debuginfo', 'rbd-fuse-debuginfo', 'rbd-mirror-debuginfo', 'rbd-nbd-debuginfo']} 2026-02-22T22:58:56.533 INFO:teuthology.task.install:extra packages: [] 2026-02-22T22:58:56.534 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': True, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': {'deb': ['python3-pytest'], 'rpm': ['python3-pytest']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'f90f4eaebb28b619cf76e80cbbe997656da69579', 'tag': None, 'wait_for_package': False} 2026-02-22T22:58:56.535 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=f90f4eaebb28b619cf76e80cbbe997656da69579 2026-02-22T22:58:56.699 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2026-02-22-1629-squid/f90f4eaebb28b619cf76e80cbbe997656da69579/centos/9/flavors/default/ 2026-02-22T22:58:56.700 INFO:teuthology.task.install.rpm:Package version is 19.2.3-689.gf90f4eae 2026-02-22T22:58:56.826 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2026-02-22-1629-squid/f90f4eaebb28b619cf76e80cbbe997656da69579/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2026-02-22-1629-squid/f90f4eaebb28b619cf76e80cbbe997656da69579/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-yuri6-testing-2026-02-22-1629-squid/f90f4eaebb28b619cf76e80cbbe997656da69579/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2026-02-22T22:58:56.826 DEBUG:teuthology.orchestra.run.trial168:> set -ex 2026-02-22T22:58:56.826 DEBUG:teuthology.orchestra.run.trial168:> sudo dd of=/etc/yum.repos.d/ceph.repo 2026-02-22T22:58:56.851 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, ceph-base-debuginfo, ceph-common-debuginfo, ceph-immutable-object-cache-debuginfo, ceph-radosgw-debuginfo, ceph-test-debuginfo, ceph-base-debuginfo, ceph-mgr-debuginfo, ceph-mds-debuginfo, ceph-mon-debuginfo, ceph-osd-debuginfo, ceph-fuse-debuginfo, librados-devel-debuginfo, libcephfs2-debuginfo, librados2-debuginfo, librbd1-debuginfo, python3-cephfs-debuginfo, python3-rados-debuginfo, python3-rbd-debuginfo, python3-rgw-debuginfo, rbd-fuse-debuginfo, rbd-mirror-debuginfo, rbd-nbd-debuginfo, python3-pytest on remote rpm x86_64 2026-02-22T22:58:56.851 DEBUG:teuthology.orchestra.run.trial168:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/f90f4eaebb28b619cf76e80cbbe997656da69579/;g' /etc/yum.repos.d/ceph.repo ; fi 2026-02-22T22:58:56.917 DEBUG:teuthology.orchestra.run.trial168:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2026-02-22T22:58:56.992 DEBUG:teuthology.orchestra.run.trial168:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2026-02-22T22:58:57.056 INFO:teuthology.orchestra.run.trial168.stdout:check_obsoletes = 1 2026-02-22T22:58:57.058 DEBUG:teuthology.orchestra.run.trial168:> sudo yum clean all 2026-02-22T22:58:57.215 INFO:teuthology.orchestra.run.trial168.stdout:41 files removed 2026-02-22T22:58:57.229 DEBUG:teuthology.orchestra.run.trial168:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd ceph-base-debuginfo ceph-common-debuginfo ceph-immutable-object-cache-debuginfo ceph-radosgw-debuginfo ceph-test-debuginfo ceph-base-debuginfo ceph-mgr-debuginfo ceph-mds-debuginfo ceph-mon-debuginfo ceph-osd-debuginfo ceph-fuse-debuginfo librados-devel-debuginfo libcephfs2-debuginfo librados2-debuginfo librbd1-debuginfo python3-cephfs-debuginfo python3-rados-debuginfo python3-rbd-debuginfo python3-rgw-debuginfo rbd-fuse-debuginfo rbd-mirror-debuginfo rbd-nbd-debuginfo python3-pytest 2026-02-22T22:58:57.501 INFO:teuthology.orchestra.run.trial168.stdout:ceph packages for x86_64 663 kB/s | 86 kB 00:00 2026-02-22T22:58:57.633 INFO:teuthology.orchestra.run.trial168.stdout:ceph noarch packages 112 kB/s | 12 kB 00:00 2026-02-22T22:58:57.762 INFO:teuthology.orchestra.run.trial168.stdout:ceph source packages 18 kB/s | 1.9 kB 00:00 2026-02-22T22:59:02.763 INFO:teuthology.orchestra.run.trial168.stdout:CentOS Stream 9 - BaseOS 1.8 MB/s | 8.9 MB 00:04 2026-02-22T22:59:05.953 INFO:teuthology.orchestra.run.trial168.stdout:CentOS Stream 9 - AppStream 10 MB/s | 27 MB 00:02 2026-02-22T22:59:09.290 INFO:teuthology.orchestra.run.trial168.stdout:CentOS Stream 9 - CRB 8.2 MB/s | 7.8 MB 00:00 2026-02-22T22:59:10.209 INFO:teuthology.orchestra.run.trial168.stdout:CentOS Stream 9 - Extras packages 106 kB/s | 20 kB 00:00 2026-02-22T22:59:10.864 INFO:teuthology.orchestra.run.trial168.stdout:Extra Packages for Enterprise Linux 34 MB/s | 20 MB 00:00 2026-02-22T22:59:14.102 INFO:teuthology.orchestra.run.trial168.stdout:lab-extras 1.4 MB/s | 50 kB 00:00 2026-02-22T22:59:15.298 INFO:teuthology.orchestra.run.trial168.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2026-02-22T22:59:15.299 INFO:teuthology.orchestra.run.trial168.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2026-02-22T22:59:15.334 INFO:teuthology.orchestra.run.trial168.stdout:Dependencies resolved. 2026-02-22T22:59:15.339 INFO:teuthology.orchestra.run.trial168.stdout:============================================================================================== 2026-02-22T22:59:15.339 INFO:teuthology.orchestra.run.trial168.stdout: Package Arch Version Repository Size 2026-02-22T22:59:15.339 INFO:teuthology.orchestra.run.trial168.stdout:============================================================================================== 2026-02-22T22:59:15.339 INFO:teuthology.orchestra.run.trial168.stdout:Installing: 2026-02-22T22:59:15.339 INFO:teuthology.orchestra.run.trial168.stdout: ceph x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 6.5 k 2026-02-22T22:59:15.340 INFO:teuthology.orchestra.run.trial168.stdout: ceph-base x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 5.5 M 2026-02-22T22:59:15.340 INFO:teuthology.orchestra.run.trial168.stdout: ceph-base-debuginfo x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 72 M 2026-02-22T22:59:15.340 INFO:teuthology.orchestra.run.trial168.stdout: ceph-common-debuginfo x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 474 M 2026-02-22T22:59:15.340 INFO:teuthology.orchestra.run.trial168.stdout: ceph-fuse x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 1.2 M 2026-02-22T22:59:15.340 INFO:teuthology.orchestra.run.trial168.stdout: ceph-fuse-debuginfo x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 6.8 M 2026-02-22T22:59:15.340 INFO:teuthology.orchestra.run.trial168.stdout: ceph-immutable-object-cache x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 145 k 2026-02-22T22:59:15.340 INFO:teuthology.orchestra.run.trial168.stdout: ceph-immutable-object-cache-debuginfo x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 1.5 M 2026-02-22T22:59:15.340 INFO:teuthology.orchestra.run.trial168.stdout: ceph-mds-debuginfo x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 55 M 2026-02-22T22:59:15.340 INFO:teuthology.orchestra.run.trial168.stdout: ceph-mgr x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 1.1 M 2026-02-22T22:59:15.340 INFO:teuthology.orchestra.run.trial168.stdout: ceph-mgr-cephadm noarch 2:19.2.3-689.gf90f4eae.el9 ceph-noarch 150 k 2026-02-22T22:59:15.340 INFO:teuthology.orchestra.run.trial168.stdout: ceph-mgr-dashboard noarch 2:19.2.3-689.gf90f4eae.el9 ceph-noarch 3.8 M 2026-02-22T22:59:15.340 INFO:teuthology.orchestra.run.trial168.stdout: ceph-mgr-debuginfo x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 18 M 2026-02-22T22:59:15.341 INFO:teuthology.orchestra.run.trial168.stdout: ceph-mgr-diskprediction-local noarch 2:19.2.3-689.gf90f4eae.el9 ceph-noarch 7.4 M 2026-02-22T22:59:15.341 INFO:teuthology.orchestra.run.trial168.stdout: ceph-mgr-rook noarch 2:19.2.3-689.gf90f4eae.el9 ceph-noarch 49 k 2026-02-22T22:59:15.341 INFO:teuthology.orchestra.run.trial168.stdout: ceph-mon-debuginfo x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 102 M 2026-02-22T22:59:15.341 INFO:teuthology.orchestra.run.trial168.stdout: ceph-osd-debuginfo x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 298 M 2026-02-22T22:59:15.341 INFO:teuthology.orchestra.run.trial168.stdout: ceph-radosgw x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 11 M 2026-02-22T22:59:15.341 INFO:teuthology.orchestra.run.trial168.stdout: ceph-radosgw-debuginfo x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 200 M 2026-02-22T22:59:15.341 INFO:teuthology.orchestra.run.trial168.stdout: ceph-test x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 50 M 2026-02-22T22:59:15.341 INFO:teuthology.orchestra.run.trial168.stdout: ceph-test-debuginfo x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 729 M 2026-02-22T22:59:15.341 INFO:teuthology.orchestra.run.trial168.stdout: ceph-volume noarch 2:19.2.3-689.gf90f4eae.el9 ceph-noarch 299 k 2026-02-22T22:59:15.341 INFO:teuthology.orchestra.run.trial168.stdout: cephadm noarch 2:19.2.3-689.gf90f4eae.el9 ceph-noarch 769 k 2026-02-22T22:59:15.341 INFO:teuthology.orchestra.run.trial168.stdout: libcephfs-devel x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 34 k 2026-02-22T22:59:15.342 INFO:teuthology.orchestra.run.trial168.stdout: libcephfs2 x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 999 k 2026-02-22T22:59:15.342 INFO:teuthology.orchestra.run.trial168.stdout: libcephfs2-debuginfo x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 6.0 M 2026-02-22T22:59:15.342 INFO:teuthology.orchestra.run.trial168.stdout: librados-devel x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 126 k 2026-02-22T22:59:15.342 INFO:teuthology.orchestra.run.trial168.stdout: librados-devel-debuginfo x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 112 k 2026-02-22T22:59:15.342 INFO:teuthology.orchestra.run.trial168.stdout: librados2-debuginfo x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 47 M 2026-02-22T22:59:15.342 INFO:teuthology.orchestra.run.trial168.stdout: librbd1-debuginfo x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 55 M 2026-02-22T22:59:15.342 INFO:teuthology.orchestra.run.trial168.stdout: python3-cephfs x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 165 k 2026-02-22T22:59:15.342 INFO:teuthology.orchestra.run.trial168.stdout: python3-cephfs-debuginfo x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 628 k 2026-02-22T22:59:15.342 INFO:teuthology.orchestra.run.trial168.stdout: python3-pytest noarch 6.2.2-7.el9 appstream 519 k 2026-02-22T22:59:15.342 INFO:teuthology.orchestra.run.trial168.stdout: python3-rados x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 322 k 2026-02-22T22:59:15.342 INFO:teuthology.orchestra.run.trial168.stdout: python3-rados-debuginfo x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 779 k 2026-02-22T22:59:15.342 INFO:teuthology.orchestra.run.trial168.stdout: python3-rbd x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 303 k 2026-02-22T22:59:15.342 INFO:teuthology.orchestra.run.trial168.stdout: python3-rbd-debuginfo x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 1.1 M 2026-02-22T22:59:15.343 INFO:teuthology.orchestra.run.trial168.stdout: python3-rgw x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 100 k 2026-02-22T22:59:15.343 INFO:teuthology.orchestra.run.trial168.stdout: python3-rgw-debuginfo x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 339 k 2026-02-22T22:59:15.343 INFO:teuthology.orchestra.run.trial168.stdout: rbd-fuse x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 85 k 2026-02-22T22:59:15.343 INFO:teuthology.orchestra.run.trial168.stdout: rbd-fuse-debuginfo x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 547 k 2026-02-22T22:59:15.343 INFO:teuthology.orchestra.run.trial168.stdout: rbd-mirror x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 3.1 M 2026-02-22T22:59:15.343 INFO:teuthology.orchestra.run.trial168.stdout: rbd-mirror-debuginfo x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 54 M 2026-02-22T22:59:15.343 INFO:teuthology.orchestra.run.trial168.stdout: rbd-nbd x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 170 k 2026-02-22T22:59:15.343 INFO:teuthology.orchestra.run.trial168.stdout: rbd-nbd-debuginfo x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 1.4 M 2026-02-22T22:59:15.343 INFO:teuthology.orchestra.run.trial168.stdout:Upgrading: 2026-02-22T22:59:15.343 INFO:teuthology.orchestra.run.trial168.stdout: librados2 x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 3.4 M 2026-02-22T22:59:15.343 INFO:teuthology.orchestra.run.trial168.stdout: librbd1 x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 3.2 M 2026-02-22T22:59:15.343 INFO:teuthology.orchestra.run.trial168.stdout:Installing dependencies: 2026-02-22T22:59:15.344 INFO:teuthology.orchestra.run.trial168.stdout: abseil-cpp x86_64 20211102.0-4.el9 epel 551 k 2026-02-22T22:59:15.344 INFO:teuthology.orchestra.run.trial168.stdout: boost-program-options x86_64 1.75.0-13.el9 appstream 104 k 2026-02-22T22:59:15.344 INFO:teuthology.orchestra.run.trial168.stdout: ceph-common x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 22 M 2026-02-22T22:59:15.344 INFO:teuthology.orchestra.run.trial168.stdout: ceph-debuginfo x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 24 M 2026-02-22T22:59:15.344 INFO:teuthology.orchestra.run.trial168.stdout: ceph-grafana-dashboards noarch 2:19.2.3-689.gf90f4eae.el9 ceph-noarch 31 k 2026-02-22T22:59:15.344 INFO:teuthology.orchestra.run.trial168.stdout: ceph-mds x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 2.4 M 2026-02-22T22:59:15.344 INFO:teuthology.orchestra.run.trial168.stdout: ceph-mgr-modules-core noarch 2:19.2.3-689.gf90f4eae.el9 ceph-noarch 253 k 2026-02-22T22:59:15.344 INFO:teuthology.orchestra.run.trial168.stdout: ceph-mon x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 4.7 M 2026-02-22T22:59:15.344 INFO:teuthology.orchestra.run.trial168.stdout: ceph-osd x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 17 M 2026-02-22T22:59:15.344 INFO:teuthology.orchestra.run.trial168.stdout: ceph-prometheus-alerts noarch 2:19.2.3-689.gf90f4eae.el9 ceph-noarch 17 k 2026-02-22T22:59:15.344 INFO:teuthology.orchestra.run.trial168.stdout: ceph-selinux x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 25 k 2026-02-22T22:59:15.344 INFO:teuthology.orchestra.run.trial168.stdout: cryptsetup x86_64 2.8.1-3.el9 baseos 351 k 2026-02-22T22:59:15.345 INFO:teuthology.orchestra.run.trial168.stdout: flexiblas x86_64 3.0.4-9.el9 appstream 30 k 2026-02-22T22:59:15.345 INFO:teuthology.orchestra.run.trial168.stdout: flexiblas-netlib x86_64 3.0.4-9.el9 appstream 3.0 M 2026-02-22T22:59:15.345 INFO:teuthology.orchestra.run.trial168.stdout: flexiblas-openblas-openmp x86_64 3.0.4-9.el9 appstream 15 k 2026-02-22T22:59:15.345 INFO:teuthology.orchestra.run.trial168.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2026-02-22T22:59:15.345 INFO:teuthology.orchestra.run.trial168.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2026-02-22T22:59:15.345 INFO:teuthology.orchestra.run.trial168.stdout: ledmon-libs x86_64 1.1.0-3.el9 baseos 40 k 2026-02-22T22:59:15.345 INFO:teuthology.orchestra.run.trial168.stdout: libarrow x86_64 9.0.0-15.el9 epel 4.4 M 2026-02-22T22:59:15.345 INFO:teuthology.orchestra.run.trial168.stdout: libarrow-doc noarch 9.0.0-15.el9 epel 25 k 2026-02-22T22:59:15.345 INFO:teuthology.orchestra.run.trial168.stdout: libcephsqlite x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 163 k 2026-02-22T22:59:15.345 INFO:teuthology.orchestra.run.trial168.stdout: libconfig x86_64 1.7.2-9.el9 baseos 72 k 2026-02-22T22:59:15.345 INFO:teuthology.orchestra.run.trial168.stdout: libgfortran x86_64 11.5.0-14.el9 baseos 794 k 2026-02-22T22:59:15.345 INFO:teuthology.orchestra.run.trial168.stdout: libnbd x86_64 1.20.3-4.el9 appstream 164 k 2026-02-22T22:59:15.345 INFO:teuthology.orchestra.run.trial168.stdout: liboath x86_64 2.6.12-1.el9 epel 49 k 2026-02-22T22:59:15.346 INFO:teuthology.orchestra.run.trial168.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2026-02-22T22:59:15.346 INFO:teuthology.orchestra.run.trial168.stdout: libquadmath x86_64 11.5.0-14.el9 baseos 184 k 2026-02-22T22:59:15.346 INFO:teuthology.orchestra.run.trial168.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2026-02-22T22:59:15.346 INFO:teuthology.orchestra.run.trial168.stdout: libradosstriper1 x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 503 k 2026-02-22T22:59:15.346 INFO:teuthology.orchestra.run.trial168.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2026-02-22T22:59:15.346 INFO:teuthology.orchestra.run.trial168.stdout: librgw2 x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 5.4 M 2026-02-22T22:59:15.346 INFO:teuthology.orchestra.run.trial168.stdout: libstoragemgmt x86_64 1.10.1-1.el9 appstream 246 k 2026-02-22T22:59:15.346 INFO:teuthology.orchestra.run.trial168.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2026-02-22T22:59:15.346 INFO:teuthology.orchestra.run.trial168.stdout: libxslt x86_64 1.1.34-12.el9 appstream 233 k 2026-02-22T22:59:15.346 INFO:teuthology.orchestra.run.trial168.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2026-02-22T22:59:15.346 INFO:teuthology.orchestra.run.trial168.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2026-02-22T22:59:15.346 INFO:teuthology.orchestra.run.trial168.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2026-02-22T22:59:15.347 INFO:teuthology.orchestra.run.trial168.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2026-02-22T22:59:15.347 INFO:teuthology.orchestra.run.trial168.stdout: mailcap noarch 2.1.49-5.el9 baseos 33 k 2026-02-22T22:59:15.347 INFO:teuthology.orchestra.run.trial168.stdout: openblas x86_64 0.3.29-1.el9 appstream 42 k 2026-02-22T22:59:15.347 INFO:teuthology.orchestra.run.trial168.stdout: openblas-openmp x86_64 0.3.29-1.el9 appstream 5.3 M 2026-02-22T22:59:15.347 INFO:teuthology.orchestra.run.trial168.stdout: parquet-libs x86_64 9.0.0-15.el9 epel 838 k 2026-02-22T22:59:15.347 INFO:teuthology.orchestra.run.trial168.stdout: pciutils x86_64 3.7.0-7.el9 baseos 93 k 2026-02-22T22:59:15.347 INFO:teuthology.orchestra.run.trial168.stdout: protobuf x86_64 3.14.0-17.el9 appstream 1.0 M 2026-02-22T22:59:15.347 INFO:teuthology.orchestra.run.trial168.stdout: protobuf-compiler x86_64 3.14.0-17.el9 crb 862 k 2026-02-22T22:59:15.347 INFO:teuthology.orchestra.run.trial168.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2026-02-22T22:59:15.347 INFO:teuthology.orchestra.run.trial168.stdout: python3-autocommand noarch 2.2.2-8.el9 epel 29 k 2026-02-22T22:59:15.347 INFO:teuthology.orchestra.run.trial168.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2026-02-22T22:59:15.347 INFO:teuthology.orchestra.run.trial168.stdout: python3-backports-tarfile noarch 1.2.0-1.el9 epel 60 k 2026-02-22T22:59:15.348 INFO:teuthology.orchestra.run.trial168.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2026-02-22T22:59:15.348 INFO:teuthology.orchestra.run.trial168.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2026-02-22T22:59:15.348 INFO:teuthology.orchestra.run.trial168.stdout: python3-ceph-argparse x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 45 k 2026-02-22T22:59:15.348 INFO:teuthology.orchestra.run.trial168.stdout: python3-ceph-common x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 142 k 2026-02-22T22:59:15.348 INFO:teuthology.orchestra.run.trial168.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2026-02-22T22:59:15.348 INFO:teuthology.orchestra.run.trial168.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2026-02-22T22:59:15.348 INFO:teuthology.orchestra.run.trial168.stdout: python3-cheroot noarch 10.0.1-4.el9 epel 173 k 2026-02-22T22:59:15.348 INFO:teuthology.orchestra.run.trial168.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2026-02-22T22:59:15.348 INFO:teuthology.orchestra.run.trial168.stdout: python3-cryptography x86_64 36.0.1-5.el9 baseos 1.2 M 2026-02-22T22:59:15.348 INFO:teuthology.orchestra.run.trial168.stdout: python3-devel x86_64 3.9.25-3.el9 appstream 244 k 2026-02-22T22:59:15.348 INFO:teuthology.orchestra.run.trial168.stdout: python3-google-auth noarch 1:2.45.0-1.el9 epel 254 k 2026-02-22T22:59:15.348 INFO:teuthology.orchestra.run.trial168.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2026-02-22T22:59:15.348 INFO:teuthology.orchestra.run.trial168.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2026-02-22T22:59:15.349 INFO:teuthology.orchestra.run.trial168.stdout: python3-iniconfig noarch 1.1.1-7.el9 appstream 17 k 2026-02-22T22:59:15.349 INFO:teuthology.orchestra.run.trial168.stdout: python3-isodate noarch 0.6.1-3.el9 epel 56 k 2026-02-22T22:59:15.349 INFO:teuthology.orchestra.run.trial168.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2026-02-22T22:59:15.349 INFO:teuthology.orchestra.run.trial168.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2026-02-22T22:59:15.349 INFO:teuthology.orchestra.run.trial168.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2026-02-22T22:59:15.349 INFO:teuthology.orchestra.run.trial168.stdout: python3-jaraco-context noarch 6.0.1-3.el9 epel 20 k 2026-02-22T22:59:15.349 INFO:teuthology.orchestra.run.trial168.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2026-02-22T22:59:15.349 INFO:teuthology.orchestra.run.trial168.stdout: python3-jaraco-text noarch 4.0.0-2.el9 epel 26 k 2026-02-22T22:59:15.349 INFO:teuthology.orchestra.run.trial168.stdout: python3-jinja2 noarch 2.11.3-8.el9 appstream 249 k 2026-02-22T22:59:15.349 INFO:teuthology.orchestra.run.trial168.stdout: python3-kubernetes noarch 1:26.1.0-3.el9 epel 1.0 M 2026-02-22T22:59:15.349 INFO:teuthology.orchestra.run.trial168.stdout: python3-libstoragemgmt x86_64 1.10.1-1.el9 appstream 177 k 2026-02-22T22:59:15.349 INFO:teuthology.orchestra.run.trial168.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2026-02-22T22:59:15.349 INFO:teuthology.orchestra.run.trial168.stdout: python3-lxml x86_64 4.6.5-3.el9 appstream 1.2 M 2026-02-22T22:59:15.350 INFO:teuthology.orchestra.run.trial168.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2026-02-22T22:59:15.350 INFO:teuthology.orchestra.run.trial168.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2026-02-22T22:59:15.350 INFO:teuthology.orchestra.run.trial168.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2026-02-22T22:59:15.350 INFO:teuthology.orchestra.run.trial168.stdout: python3-msgpack x86_64 1.0.3-2.el9 epel 86 k 2026-02-22T22:59:15.350 INFO:teuthology.orchestra.run.trial168.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2026-02-22T22:59:15.350 INFO:teuthology.orchestra.run.trial168.stdout: python3-numpy x86_64 1:1.23.5-2.el9 appstream 6.1 M 2026-02-22T22:59:15.350 INFO:teuthology.orchestra.run.trial168.stdout: python3-numpy-f2py x86_64 1:1.23.5-2.el9 appstream 442 k 2026-02-22T22:59:15.350 INFO:teuthology.orchestra.run.trial168.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2026-02-22T22:59:15.350 INFO:teuthology.orchestra.run.trial168.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2026-02-22T22:59:15.350 INFO:teuthology.orchestra.run.trial168.stdout: python3-pluggy noarch 0.13.1-7.el9 appstream 41 k 2026-02-22T22:59:15.350 INFO:teuthology.orchestra.run.trial168.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2026-02-22T22:59:15.350 INFO:teuthology.orchestra.run.trial168.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2026-02-22T22:59:15.350 INFO:teuthology.orchestra.run.trial168.stdout: python3-protobuf noarch 3.14.0-17.el9 appstream 267 k 2026-02-22T22:59:15.351 INFO:teuthology.orchestra.run.trial168.stdout: python3-py noarch 1.10.0-6.el9 appstream 477 k 2026-02-22T22:59:15.351 INFO:teuthology.orchestra.run.trial168.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2026-02-22T22:59:15.351 INFO:teuthology.orchestra.run.trial168.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2026-02-22T22:59:15.351 INFO:teuthology.orchestra.run.trial168.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2026-02-22T22:59:15.351 INFO:teuthology.orchestra.run.trial168.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2026-02-22T22:59:15.351 INFO:teuthology.orchestra.run.trial168.stdout: python3-pyparsing noarch 2.4.7-9.el9 baseos 150 k 2026-02-22T22:59:15.351 INFO:teuthology.orchestra.run.trial168.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2026-02-22T22:59:15.351 INFO:teuthology.orchestra.run.trial168.stdout: python3-requests noarch 2.25.1-10.el9 baseos 126 k 2026-02-22T22:59:15.351 INFO:teuthology.orchestra.run.trial168.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2026-02-22T22:59:15.351 INFO:teuthology.orchestra.run.trial168.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2026-02-22T22:59:15.351 INFO:teuthology.orchestra.run.trial168.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2026-02-22T22:59:15.351 INFO:teuthology.orchestra.run.trial168.stdout: python3-scipy x86_64 1.9.3-2.el9 appstream 19 M 2026-02-22T22:59:15.351 INFO:teuthology.orchestra.run.trial168.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2026-02-22T22:59:15.352 INFO:teuthology.orchestra.run.trial168.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2026-02-22T22:59:15.352 INFO:teuthology.orchestra.run.trial168.stdout: python3-typing-extensions noarch 4.15.0-1.el9 epel 86 k 2026-02-22T22:59:15.352 INFO:teuthology.orchestra.run.trial168.stdout: python3-urllib3 noarch 1.26.5-7.el9 baseos 218 k 2026-02-22T22:59:15.352 INFO:teuthology.orchestra.run.trial168.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2026-02-22T22:59:15.352 INFO:teuthology.orchestra.run.trial168.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2026-02-22T22:59:15.352 INFO:teuthology.orchestra.run.trial168.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2026-02-22T22:59:15.352 INFO:teuthology.orchestra.run.trial168.stdout: python3-xmlsec x86_64 1.3.13-1.el9 epel 48 k 2026-02-22T22:59:15.352 INFO:teuthology.orchestra.run.trial168.stdout: python3-xmltodict noarch 0.12.0-15.el9 epel 22 k 2026-02-22T22:59:15.352 INFO:teuthology.orchestra.run.trial168.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2026-02-22T22:59:15.352 INFO:teuthology.orchestra.run.trial168.stdout: qatlib x86_64 25.08.0-2.el9 appstream 240 k 2026-02-22T22:59:15.352 INFO:teuthology.orchestra.run.trial168.stdout: qatzip-libs x86_64 1.3.1-1.el9 appstream 66 k 2026-02-22T22:59:15.352 INFO:teuthology.orchestra.run.trial168.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2026-02-22T22:59:15.352 INFO:teuthology.orchestra.run.trial168.stdout: socat x86_64 1.7.4.1-8.el9 appstream 303 k 2026-02-22T22:59:15.353 INFO:teuthology.orchestra.run.trial168.stdout: thrift x86_64 0.15.0-4.el9 epel 1.6 M 2026-02-22T22:59:15.353 INFO:teuthology.orchestra.run.trial168.stdout: unzip x86_64 6.0-59.el9 baseos 182 k 2026-02-22T22:59:15.353 INFO:teuthology.orchestra.run.trial168.stdout: xmlsec1 x86_64 1.2.29-13.el9 appstream 189 k 2026-02-22T22:59:15.353 INFO:teuthology.orchestra.run.trial168.stdout: xmlsec1-openssl x86_64 1.2.29-13.el9 appstream 90 k 2026-02-22T22:59:15.353 INFO:teuthology.orchestra.run.trial168.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2026-02-22T22:59:15.353 INFO:teuthology.orchestra.run.trial168.stdout: zip x86_64 3.0-35.el9 baseos 266 k 2026-02-22T22:59:15.353 INFO:teuthology.orchestra.run.trial168.stdout:Installing weak dependencies: 2026-02-22T22:59:15.353 INFO:teuthology.orchestra.run.trial168.stdout: ceph-debugsource x86_64 2:19.2.3-689.gf90f4eae.el9 ceph 11 M 2026-02-22T22:59:15.353 INFO:teuthology.orchestra.run.trial168.stdout: ceph-mgr-k8sevents noarch 2:19.2.3-689.gf90f4eae.el9 ceph-noarch 22 k 2026-02-22T22:59:15.353 INFO:teuthology.orchestra.run.trial168.stdout: nvme-cli x86_64 2.16-1.el9 baseos 1.2 M 2026-02-22T22:59:15.353 INFO:teuthology.orchestra.run.trial168.stdout: python3-influxdb noarch 5.3.1-1.el9 epel 139 k 2026-02-22T22:59:15.353 INFO:teuthology.orchestra.run.trial168.stdout: python3-saml noarch 1.16.0-1.el9 epel 125 k 2026-02-22T22:59:15.354 INFO:teuthology.orchestra.run.trial168.stdout: qatlib-service x86_64 25.08.0-2.el9 appstream 37 k 2026-02-22T22:59:15.354 INFO:teuthology.orchestra.run.trial168.stdout: 2026-02-22T22:59:15.354 INFO:teuthology.orchestra.run.trial168.stdout:Transaction Summary 2026-02-22T22:59:15.354 INFO:teuthology.orchestra.run.trial168.stdout:============================================================================================== 2026-02-22T22:59:15.354 INFO:teuthology.orchestra.run.trial168.stdout:Install 171 Packages 2026-02-22T22:59:15.354 INFO:teuthology.orchestra.run.trial168.stdout:Upgrade 2 Packages 2026-02-22T22:59:15.354 INFO:teuthology.orchestra.run.trial168.stdout: 2026-02-22T22:59:15.355 INFO:teuthology.orchestra.run.trial168.stdout:Total download size: 2.3 G 2026-02-22T22:59:15.355 INFO:teuthology.orchestra.run.trial168.stdout:Downloading Packages: 2026-02-22T22:59:15.747 INFO:teuthology.orchestra.run.trial168.stdout:(1/173): ceph-19.2.3-689.gf90f4eae.el9.x86_64.r 127 kB/s | 6.5 kB 00:00 2026-02-22T22:59:15.992 INFO:teuthology.orchestra.run.trial168.stdout:(2/173): ceph-base-19.2.3-689.gf90f4eae.el9.x86 19 MB/s | 5.5 MB 00:00 2026-02-22T22:59:17.215 INFO:teuthology.orchestra.run.trial168.stdout:(3/173): ceph-common-19.2.3-689.gf90f4eae.el9.x 15 MB/s | 22 MB 00:01 2026-02-22T22:59:18.846 INFO:teuthology.orchestra.run.trial168.stdout:(4/173): ceph-debuginfo-19.2.3-689.gf90f4eae.el 15 MB/s | 24 MB 00:01 2026-02-22T22:59:19.641 INFO:teuthology.orchestra.run.trial168.stdout:(5/173): ceph-base-debuginfo-19.2.3-689.gf90f4e 18 MB/s | 72 MB 00:03 2026-02-22T22:59:19.655 INFO:teuthology.orchestra.run.trial168.stdout:(6/173): ceph-debugsource-19.2.3-689.gf90f4eae. 13 MB/s | 11 MB 00:00 2026-02-22T22:59:19.682 INFO:teuthology.orchestra.run.trial168.stdout:(7/173): ceph-fuse-19.2.3-689.gf90f4eae.el9.x86 28 MB/s | 1.2 MB 00:00 2026-02-22T22:59:19.740 INFO:teuthology.orchestra.run.trial168.stdout:(8/173): ceph-immutable-object-cache-19.2.3-689 2.5 MB/s | 145 kB 00:00 2026-02-22T22:59:19.968 INFO:teuthology.orchestra.run.trial168.stdout:(9/173): ceph-immutable-object-cache-debuginfo- 6.7 MB/s | 1.5 MB 00:00 2026-02-22T22:59:20.046 INFO:teuthology.orchestra.run.trial168.stdout:(10/173): ceph-fuse-debuginfo-19.2.3-689.gf90f4 18 MB/s | 6.8 MB 00:00 2026-02-22T22:59:20.144 INFO:teuthology.orchestra.run.trial168.stdout:(11/173): ceph-mds-19.2.3-689.gf90f4eae.el9.x86 14 MB/s | 2.4 MB 00:00 2026-02-22T22:59:20.259 INFO:teuthology.orchestra.run.trial168.stdout:(12/173): ceph-mgr-19.2.3-689.gf90f4eae.el9.x86 9.4 MB/s | 1.1 MB 00:00 2026-02-22T22:59:21.514 INFO:teuthology.orchestra.run.trial168.stdout:(13/173): ceph-mgr-debuginfo-19.2.3-689.gf90f4e 14 MB/s | 18 MB 00:01 2026-02-22T22:59:21.846 INFO:teuthology.orchestra.run.trial168.stdout:(14/173): ceph-mon-19.2.3-689.gf90f4eae.el9.x86 14 MB/s | 4.7 MB 00:00 2026-02-22T22:59:23.387 INFO:teuthology.orchestra.run.trial168.stdout:(15/173): ceph-mds-debuginfo-19.2.3-689.gf90f4e 17 MB/s | 55 MB 00:03 2026-02-22T22:59:24.438 INFO:teuthology.orchestra.run.trial168.stdout:(16/173): ceph-osd-19.2.3-689.gf90f4eae.el9.x86 16 MB/s | 17 MB 00:01 2026-02-22T22:59:28.297 INFO:teuthology.orchestra.run.trial168.stdout:(17/173): ceph-mon-debuginfo-19.2.3-689.gf90f4e 16 MB/s | 102 MB 00:06 2026-02-22T22:59:29.073 INFO:teuthology.orchestra.run.trial168.stdout:(18/173): ceph-radosgw-19.2.3-689.gf90f4eae.el9 14 MB/s | 11 MB 00:00 2026-02-22T22:59:39.048 INFO:teuthology.orchestra.run.trial168.stdout:(19/173): ceph-common-debuginfo-19.2.3-689.gf90 21 MB/s | 474 MB 00:23 2026-02-22T22:59:39.070 INFO:teuthology.orchestra.run.trial168.stdout:(20/173): ceph-selinux-19.2.3-689.gf90f4eae.el9 1.1 MB/s | 25 kB 00:00 2026-02-22T22:59:42.599 INFO:teuthology.orchestra.run.trial168.stdout:(21/173): ceph-radosgw-debuginfo-19.2.3-689.gf9 15 MB/s | 200 MB 00:13 2026-02-22T22:59:43.738 INFO:teuthology.orchestra.run.trial168.stdout:(22/173): ceph-test-19.2.3-689.gf90f4eae.el9.x8 11 MB/s | 50 MB 00:04 2026-02-22T22:59:43.791 INFO:teuthology.orchestra.run.trial168.stdout:(23/173): libcephfs-devel-19.2.3-689.gf90f4eae. 642 kB/s | 34 kB 00:00 2026-02-22T22:59:43.913 INFO:teuthology.orchestra.run.trial168.stdout:(24/173): libcephfs2-19.2.3-689.gf90f4eae.el9.x 8.0 MB/s | 999 kB 00:00 2026-02-22T22:59:44.606 INFO:teuthology.orchestra.run.trial168.stdout:(25/173): libcephfs2-debuginfo-19.2.3-689.gf90f 8.6 MB/s | 6.0 MB 00:00 2026-02-22T22:59:44.655 INFO:teuthology.orchestra.run.trial168.stdout:(26/173): libcephsqlite-19.2.3-689.gf90f4eae.el 3.2 MB/s | 163 kB 00:00 2026-02-22T22:59:44.704 INFO:teuthology.orchestra.run.trial168.stdout:(27/173): librados-devel-19.2.3-689.gf90f4eae.e 2.5 MB/s | 126 kB 00:00 2026-02-22T22:59:44.753 INFO:teuthology.orchestra.run.trial168.stdout:(28/173): librados-devel-debuginfo-19.2.3-689.g 2.2 MB/s | 112 kB 00:00 2026-02-22T22:59:48.900 INFO:teuthology.orchestra.run.trial168.stdout:(29/173): ceph-osd-debuginfo-19.2.3-689.gf90f4e 12 MB/s | 298 MB 00:24 2026-02-22T22:59:48.960 INFO:teuthology.orchestra.run.trial168.stdout:(30/173): libradosstriper1-19.2.3-689.gf90f4eae 8.2 MB/s | 503 kB 00:00 2026-02-22T22:59:49.142 INFO:teuthology.orchestra.run.trial168.stdout:(31/173): librados2-debuginfo-19.2.3-689.gf90f4 11 MB/s | 47 MB 00:04 2026-02-22T22:59:49.626 INFO:teuthology.orchestra.run.trial168.stdout:(32/173): librgw2-19.2.3-689.gf90f4eae.el9.x86_ 11 MB/s | 5.4 MB 00:00 2026-02-22T22:59:49.667 INFO:teuthology.orchestra.run.trial168.stdout:(33/173): python3-ceph-argparse-19.2.3-689.gf90 1.1 MB/s | 45 kB 00:00 2026-02-22T22:59:49.710 INFO:teuthology.orchestra.run.trial168.stdout:(34/173): python3-ceph-common-19.2.3-689.gf90f4 3.2 MB/s | 142 kB 00:00 2026-02-22T22:59:49.754 INFO:teuthology.orchestra.run.trial168.stdout:(35/173): python3-cephfs-19.2.3-689.gf90f4eae.e 3.7 MB/s | 165 kB 00:00 2026-02-22T22:59:49.845 INFO:teuthology.orchestra.run.trial168.stdout:(36/173): python3-cephfs-debuginfo-19.2.3-689.g 6.7 MB/s | 628 kB 00:00 2026-02-22T22:59:49.891 INFO:teuthology.orchestra.run.trial168.stdout:(37/173): python3-rados-19.2.3-689.gf90f4eae.el 6.8 MB/s | 322 kB 00:00 2026-02-22T22:59:49.986 INFO:teuthology.orchestra.run.trial168.stdout:(38/173): python3-rados-debuginfo-19.2.3-689.gf 8.1 MB/s | 779 kB 00:00 2026-02-22T22:59:50.032 INFO:teuthology.orchestra.run.trial168.stdout:(39/173): python3-rbd-19.2.3-689.gf90f4eae.el9. 6.4 MB/s | 303 kB 00:00 2026-02-22T22:59:50.134 INFO:teuthology.orchestra.run.trial168.stdout:(40/173): python3-rbd-debuginfo-19.2.3-689.gf90 11 MB/s | 1.1 MB 00:00 2026-02-22T22:59:50.177 INFO:teuthology.orchestra.run.trial168.stdout:(41/173): python3-rgw-19.2.3-689.gf90f4eae.el9. 2.3 MB/s | 100 kB 00:00 2026-02-22T22:59:50.224 INFO:teuthology.orchestra.run.trial168.stdout:(42/173): python3-rgw-debuginfo-19.2.3-689.gf90 7.0 MB/s | 339 kB 00:00 2026-02-22T22:59:50.267 INFO:teuthology.orchestra.run.trial168.stdout:(43/173): rbd-fuse-19.2.3-689.gf90f4eae.el9.x86 2.0 MB/s | 85 kB 00:00 2026-02-22T22:59:50.318 INFO:teuthology.orchestra.run.trial168.stdout:(44/173): rbd-fuse-debuginfo-19.2.3-689.gf90f4e 10 MB/s | 547 kB 00:00 2026-02-22T22:59:50.617 INFO:teuthology.orchestra.run.trial168.stdout:(45/173): rbd-mirror-19.2.3-689.gf90f4eae.el9.x 10 MB/s | 3.1 MB 00:00 2026-02-22T22:59:52.997 INFO:teuthology.orchestra.run.trial168.stdout:(46/173): librbd1-debuginfo-19.2.3-689.gf90f4ea 14 MB/s | 55 MB 00:04 2026-02-22T22:59:53.043 INFO:teuthology.orchestra.run.trial168.stdout:(47/173): rbd-nbd-19.2.3-689.gf90f4eae.el9.x86_ 3.6 MB/s | 170 kB 00:00 2026-02-22T22:59:53.154 INFO:teuthology.orchestra.run.trial168.stdout:(48/173): rbd-nbd-debuginfo-19.2.3-689.gf90f4ea 12 MB/s | 1.4 MB 00:00 2026-02-22T22:59:53.197 INFO:teuthology.orchestra.run.trial168.stdout:(49/173): ceph-grafana-dashboards-19.2.3-689.gf 723 kB/s | 31 kB 00:00 2026-02-22T22:59:53.242 INFO:teuthology.orchestra.run.trial168.stdout:(50/173): ceph-mgr-cephadm-19.2.3-689.gf90f4eae 3.3 MB/s | 150 kB 00:00 2026-02-22T22:59:53.567 INFO:teuthology.orchestra.run.trial168.stdout:(51/173): ceph-mgr-dashboard-19.2.3-689.gf90f4e 12 MB/s | 3.8 MB 00:00 2026-02-22T22:59:54.190 INFO:teuthology.orchestra.run.trial168.stdout:(52/173): ceph-mgr-diskprediction-local-19.2.3- 12 MB/s | 7.4 MB 00:00 2026-02-22T22:59:54.232 INFO:teuthology.orchestra.run.trial168.stdout:(53/173): ceph-mgr-k8sevents-19.2.3-689.gf90f4e 519 kB/s | 22 kB 00:00 2026-02-22T22:59:54.273 INFO:teuthology.orchestra.run.trial168.stdout:(54/173): ceph-mgr-modules-core-19.2.3-689.gf90 6.0 MB/s | 253 kB 00:00 2026-02-22T22:59:54.309 INFO:teuthology.orchestra.run.trial168.stdout:(55/173): ceph-mgr-rook-19.2.3-689.gf90f4eae.el 1.3 MB/s | 49 kB 00:00 2026-02-22T22:59:54.345 INFO:teuthology.orchestra.run.trial168.stdout:(56/173): ceph-prometheus-alerts-19.2.3-689.gf9 469 kB/s | 17 kB 00:00 2026-02-22T22:59:54.386 INFO:teuthology.orchestra.run.trial168.stdout:(57/173): ceph-volume-19.2.3-689.gf90f4eae.el9. 7.1 MB/s | 299 kB 00:00 2026-02-22T22:59:54.471 INFO:teuthology.orchestra.run.trial168.stdout:(58/173): cephadm-19.2.3-689.gf90f4eae.el9.noar 8.9 MB/s | 769 kB 00:00 2026-02-22T22:59:54.561 INFO:teuthology.orchestra.run.trial168.stdout:(59/173): cryptsetup-2.8.1-3.el9.x86_64.rpm 3.8 MB/s | 351 kB 00:00 2026-02-22T22:59:54.581 INFO:teuthology.orchestra.run.trial168.stdout:(60/173): ledmon-libs-1.1.0-3.el9.x86_64.rpm 1.9 MB/s | 40 kB 00:00 2026-02-22T22:59:54.607 INFO:teuthology.orchestra.run.trial168.stdout:(61/173): libconfig-1.7.2-9.el9.x86_64.rpm 2.8 MB/s | 72 kB 00:00 2026-02-22T22:59:54.643 INFO:teuthology.orchestra.run.trial168.stdout:(62/173): libgfortran-11.5.0-14.el9.x86_64.rpm 21 MB/s | 794 kB 00:00 2026-02-22T22:59:54.667 INFO:teuthology.orchestra.run.trial168.stdout:(63/173): libquadmath-11.5.0-14.el9.x86_64.rpm 7.7 MB/s | 184 kB 00:00 2026-02-22T22:59:54.691 INFO:teuthology.orchestra.run.trial168.stdout:(64/173): mailcap-2.1.49-5.el9.noarch.rpm 1.4 MB/s | 33 kB 00:00 2026-02-22T22:59:54.723 INFO:teuthology.orchestra.run.trial168.stdout:(65/173): nvme-cli-2.16-1.el9.x86_64.rpm 36 MB/s | 1.2 MB 00:00 2026-02-22T22:59:54.745 INFO:teuthology.orchestra.run.trial168.stdout:(66/173): pciutils-3.7.0-7.el9.x86_64.rpm 4.1 MB/s | 93 kB 00:00 2026-02-22T22:59:54.768 INFO:teuthology.orchestra.run.trial168.stdout:(67/173): python3-cffi-1.14.5-5.el9.x86_64.rpm 11 MB/s | 253 kB 00:00 2026-02-22T22:59:54.792 INFO:teuthology.orchestra.run.trial168.stdout:(68/173): python3-cryptography-36.0.1-5.el9.x86 51 MB/s | 1.2 MB 00:00 2026-02-22T22:59:54.814 INFO:teuthology.orchestra.run.trial168.stdout:(69/173): python3-ply-3.11-14.el9.noarch.rpm 5.0 MB/s | 106 kB 00:00 2026-02-22T22:59:54.840 INFO:teuthology.orchestra.run.trial168.stdout:(70/173): python3-pycparser-2.20-6.el9.noarch.r 5.1 MB/s | 135 kB 00:00 2026-02-22T22:59:54.866 INFO:teuthology.orchestra.run.trial168.stdout:(71/173): python3-pyparsing-2.4.7-9.el9.noarch. 5.6 MB/s | 150 kB 00:00 2026-02-22T22:59:54.890 INFO:teuthology.orchestra.run.trial168.stdout:(72/173): python3-requests-2.25.1-10.el9.noarch 5.3 MB/s | 126 kB 00:00 2026-02-22T22:59:54.918 INFO:teuthology.orchestra.run.trial168.stdout:(73/173): python3-urllib3-1.26.5-7.el9.noarch.r 7.5 MB/s | 218 kB 00:00 2026-02-22T22:59:55.011 INFO:teuthology.orchestra.run.trial168.stdout:(74/173): rbd-mirror-debuginfo-19.2.3-689.gf90f 12 MB/s | 54 MB 00:04 2026-02-22T22:59:55.012 INFO:teuthology.orchestra.run.trial168.stdout:(75/173): unzip-6.0-59.el9.x86_64.rpm 1.9 MB/s | 182 kB 00:00 2026-02-22T22:59:55.040 INFO:teuthology.orchestra.run.trial168.stdout:(76/173): zip-3.0-35.el9.x86_64.rpm 9.2 MB/s | 266 kB 00:00 2026-02-22T22:59:55.081 INFO:teuthology.orchestra.run.trial168.stdout:(77/173): boost-program-options-1.75.0-13.el9.x 1.5 MB/s | 104 kB 00:00 2026-02-22T22:59:55.082 INFO:teuthology.orchestra.run.trial168.stdout:(78/173): flexiblas-3.0.4-9.el9.x86_64.rpm 701 kB/s | 30 kB 00:00 2026-02-22T22:59:55.104 INFO:teuthology.orchestra.run.trial168.stdout:(79/173): flexiblas-openblas-openmp-3.0.4-9.el9 693 kB/s | 15 kB 00:00 2026-02-22T22:59:55.148 INFO:teuthology.orchestra.run.trial168.stdout:(80/173): libnbd-1.20.3-4.el9.x86_64.rpm 3.7 MB/s | 164 kB 00:00 2026-02-22T22:59:55.172 INFO:teuthology.orchestra.run.trial168.stdout:(81/173): libpmemobj-1.12.1-1.el9.x86_64.rpm 6.3 MB/s | 160 kB 00:00 2026-02-22T22:59:55.191 INFO:teuthology.orchestra.run.trial168.stdout:(82/173): flexiblas-netlib-3.0.4-9.el9.x86_64.r 27 MB/s | 3.0 MB 00:00 2026-02-22T22:59:55.194 INFO:teuthology.orchestra.run.trial168.stdout:(83/173): librabbitmq-0.11.0-7.el9.x86_64.rpm 2.0 MB/s | 45 kB 00:00 2026-02-22T22:59:55.216 INFO:teuthology.orchestra.run.trial168.stdout:(84/173): librdkafka-1.6.1-102.el9.x86_64.rpm 26 MB/s | 662 kB 00:00 2026-02-22T22:59:55.218 INFO:teuthology.orchestra.run.trial168.stdout:(85/173): libstoragemgmt-1.10.1-1.el9.x86_64.rp 10 MB/s | 246 kB 00:00 2026-02-22T22:59:55.240 INFO:teuthology.orchestra.run.trial168.stdout:(86/173): libxslt-1.1.34-12.el9.x86_64.rpm 9.6 MB/s | 233 kB 00:00 2026-02-22T22:59:55.245 INFO:teuthology.orchestra.run.trial168.stdout:(87/173): lttng-ust-2.12.0-6.el9.x86_64.rpm 11 MB/s | 292 kB 00:00 2026-02-22T22:59:55.263 INFO:teuthology.orchestra.run.trial168.stdout:(88/173): lua-5.4.4-4.el9.x86_64.rpm 7.8 MB/s | 188 kB 00:00 2026-02-22T22:59:55.267 INFO:teuthology.orchestra.run.trial168.stdout:(89/173): openblas-0.3.29-1.el9.x86_64.rpm 1.9 MB/s | 42 kB 00:00 2026-02-22T22:59:55.314 INFO:teuthology.orchestra.run.trial168.stdout:(90/173): protobuf-3.14.0-17.el9.x86_64.rpm 21 MB/s | 1.0 MB 00:00 2026-02-22T22:59:55.336 INFO:teuthology.orchestra.run.trial168.stdout:(91/173): openblas-openmp-0.3.29-1.el9.x86_64.r 72 MB/s | 5.3 MB 00:00 2026-02-22T22:59:55.483 INFO:teuthology.orchestra.run.trial168.stdout:(92/173): python3-devel-3.9.25-3.el9.x86_64.rpm 1.6 MB/s | 244 kB 00:00 2026-02-22T22:59:55.505 INFO:teuthology.orchestra.run.trial168.stdout:(93/173): python3-iniconfig-1.1.1-7.el9.noarch. 774 kB/s | 17 kB 00:00 2026-02-22T22:59:55.529 INFO:teuthology.orchestra.run.trial168.stdout:(94/173): python3-jinja2-2.11.3-8.el9.noarch.rp 10 MB/s | 249 kB 00:00 2026-02-22T22:59:55.554 INFO:teuthology.orchestra.run.trial168.stdout:(95/173): python3-libstoragemgmt-1.10.1-1.el9.x 7.2 MB/s | 177 kB 00:00 2026-02-22T22:59:55.601 INFO:teuthology.orchestra.run.trial168.stdout:(96/173): python3-lxml-4.6.5-3.el9.x86_64.rpm 26 MB/s | 1.2 MB 00:00 2026-02-22T22:59:55.626 INFO:teuthology.orchestra.run.trial168.stdout:(97/173): python3-mako-1.1.4-6.el9.noarch.rpm 6.9 MB/s | 172 kB 00:00 2026-02-22T22:59:55.648 INFO:teuthology.orchestra.run.trial168.stdout:(98/173): python3-markupsafe-1.1.1-12.el9.x86_6 1.5 MB/s | 35 kB 00:00 2026-02-22T22:59:55.693 INFO:teuthology.orchestra.run.trial168.stdout:(99/173): python3-babel-2.9.1-2.el9.noarch.rpm 16 MB/s | 6.0 MB 00:00 2026-02-22T22:59:55.719 INFO:teuthology.orchestra.run.trial168.stdout:(100/173): python3-numpy-f2py-1.23.5-2.el9.x86_ 17 MB/s | 442 kB 00:00 2026-02-22T22:59:55.742 INFO:teuthology.orchestra.run.trial168.stdout:(101/173): python3-numpy-1.23.5-2.el9.x86_64.rp 65 MB/s | 6.1 MB 00:00 2026-02-22T22:59:55.743 INFO:teuthology.orchestra.run.trial168.stdout:(102/173): python3-packaging-20.9-5.el9.noarch. 3.2 MB/s | 77 kB 00:00 2026-02-22T22:59:55.765 INFO:teuthology.orchestra.run.trial168.stdout:(103/173): python3-pluggy-0.13.1-7.el9.noarch.r 1.8 MB/s | 41 kB 00:00 2026-02-22T22:59:55.767 INFO:teuthology.orchestra.run.trial168.stdout:(104/173): python3-protobuf-3.14.0-17.el9.noarc 11 MB/s | 267 kB 00:00 2026-02-22T22:59:55.791 INFO:teuthology.orchestra.run.trial168.stdout:(105/173): python3-py-1.10.0-6.el9.noarch.rpm 18 MB/s | 477 kB 00:00 2026-02-22T22:59:55.793 INFO:teuthology.orchestra.run.trial168.stdout:(106/173): python3-pyasn1-0.4.8-6.el9.noarch.rp 6.0 MB/s | 159 kB 00:00 2026-02-22T22:59:55.816 INFO:teuthology.orchestra.run.trial168.stdout:(107/173): python3-pyasn1-modules-0.4.8-6.el9.n 11 MB/s | 279 kB 00:00 2026-02-22T22:59:55.819 INFO:teuthology.orchestra.run.trial168.stdout:(108/173): python3-pytest-6.2.2-7.el9.noarch.rp 19 MB/s | 519 kB 00:00 2026-02-22T22:59:55.839 INFO:teuthology.orchestra.run.trial168.stdout:(109/173): python3-requests-oauthlib-1.3.0-12.e 2.3 MB/s | 54 kB 00:00 2026-02-22T22:59:55.861 INFO:teuthology.orchestra.run.trial168.stdout:(110/173): python3-toml-0.10.2-6.el9.noarch.rpm 1.8 MB/s | 42 kB 00:00 2026-02-22T22:59:55.883 INFO:teuthology.orchestra.run.trial168.stdout:(111/173): qatlib-25.08.0-2.el9.x86_64.rpm 11 MB/s | 240 kB 00:00 2026-02-22T22:59:55.905 INFO:teuthology.orchestra.run.trial168.stdout:(112/173): qatlib-service-25.08.0-2.el9.x86_64. 1.7 MB/s | 37 kB 00:00 2026-02-22T22:59:55.927 INFO:teuthology.orchestra.run.trial168.stdout:(113/173): qatzip-libs-1.3.1-1.el9.x86_64.rpm 3.0 MB/s | 66 kB 00:00 2026-02-22T22:59:55.950 INFO:teuthology.orchestra.run.trial168.stdout:(114/173): socat-1.7.4.1-8.el9.x86_64.rpm 13 MB/s | 303 kB 00:00 2026-02-22T22:59:55.974 INFO:teuthology.orchestra.run.trial168.stdout:(115/173): xmlsec1-1.2.29-13.el9.x86_64.rpm 7.8 MB/s | 189 kB 00:00 2026-02-22T22:59:55.996 INFO:teuthology.orchestra.run.trial168.stdout:(116/173): xmlsec1-openssl-1.2.29-13.el9.x86_64 3.9 MB/s | 90 kB 00:00 2026-02-22T22:59:56.018 INFO:teuthology.orchestra.run.trial168.stdout:(117/173): xmlstarlet-1.6.1-20.el9.x86_64.rpm 2.8 MB/s | 64 kB 00:00 2026-02-22T22:59:56.064 INFO:teuthology.orchestra.run.trial168.stdout:(118/173): lua-devel-5.4.4-4.el9.x86_64.rpm 502 kB/s | 22 kB 00:00 2026-02-22T22:59:56.109 INFO:teuthology.orchestra.run.trial168.stdout:(119/173): protobuf-compiler-3.14.0-17.el9.x86_ 19 MB/s | 862 kB 00:00 2026-02-22T22:59:56.313 INFO:teuthology.orchestra.run.trial168.stdout:(120/173): abseil-cpp-20211102.0-4.el9.x86_64.r 2.6 MB/s | 551 kB 00:00 2026-02-22T22:59:56.344 INFO:teuthology.orchestra.run.trial168.stdout:(121/173): gperftools-libs-2.9.1-3.el9.x86_64.r 9.7 MB/s | 308 kB 00:00 2026-02-22T22:59:56.372 INFO:teuthology.orchestra.run.trial168.stdout:(122/173): grpc-data-1.46.7-10.el9.noarch.rpm 682 kB/s | 19 kB 00:00 2026-02-22T22:59:56.507 INFO:teuthology.orchestra.run.trial168.stdout:(123/173): python3-scipy-1.9.3-2.el9.x86_64.rpm 28 MB/s | 19 MB 00:00 2026-02-22T22:59:56.569 INFO:teuthology.orchestra.run.trial168.stdout:(124/173): libarrow-9.0.0-15.el9.x86_64.rpm 22 MB/s | 4.4 MB 00:00 2026-02-22T22:59:56.595 INFO:teuthology.orchestra.run.trial168.stdout:(125/173): libarrow-doc-9.0.0-15.el9.noarch.rpm 285 kB/s | 25 kB 00:00 2026-02-22T22:59:56.598 INFO:teuthology.orchestra.run.trial168.stdout:(126/173): liboath-2.6.12-1.el9.x86_64.rpm 1.6 MB/s | 49 kB 00:00 2026-02-22T22:59:56.632 INFO:teuthology.orchestra.run.trial168.stdout:(127/173): luarocks-3.9.2-5.el9.noarch.rpm 4.4 MB/s | 151 kB 00:00 2026-02-22T22:59:56.653 INFO:teuthology.orchestra.run.trial168.stdout:(128/173): libunwind-1.6.2-1.el9.x86_64.rpm 1.1 MB/s | 67 kB 00:00 2026-02-22T22:59:56.664 INFO:teuthology.orchestra.run.trial168.stdout:(129/173): parquet-libs-9.0.0-15.el9.x86_64.rpm 25 MB/s | 838 kB 00:00 2026-02-22T22:59:56.693 INFO:teuthology.orchestra.run.trial168.stdout:(130/173): python3-autocommand-2.2.2-8.el9.noar 1.0 MB/s | 29 kB 00:00 2026-02-22T22:59:56.723 INFO:teuthology.orchestra.run.trial168.stdout:(131/173): python3-backports-tarfile-1.2.0-1.el 2.0 MB/s | 60 kB 00:00 2026-02-22T22:59:56.743 INFO:teuthology.orchestra.run.trial168.stdout:(132/173): python3-asyncssh-2.13.2-5.el9.noarch 6.0 MB/s | 548 kB 00:00 2026-02-22T22:59:56.752 INFO:teuthology.orchestra.run.trial168.stdout:(133/173): python3-bcrypt-3.2.2-1.el9.x86_64.rp 1.5 MB/s | 43 kB 00:00 2026-02-22T22:59:56.773 INFO:teuthology.orchestra.run.trial168.stdout:(134/173): python3-cachetools-4.2.4-1.el9.noarc 1.0 MB/s | 32 kB 00:00 2026-02-22T22:59:56.790 INFO:teuthology.orchestra.run.trial168.stdout:(135/173): python3-certifi-2023.05.07-4.el9.noa 371 kB/s | 14 kB 00:00 2026-02-22T22:59:56.805 INFO:teuthology.orchestra.run.trial168.stdout:(136/173): python3-cheroot-10.0.1-4.el9.noarch. 5.3 MB/s | 173 kB 00:00 2026-02-22T22:59:56.822 INFO:teuthology.orchestra.run.trial168.stdout:(137/173): python3-cherrypy-18.6.1-2.el9.noarch 11 MB/s | 358 kB 00:00 2026-02-22T22:59:56.838 INFO:teuthology.orchestra.run.trial168.stdout:(138/173): python3-google-auth-2.45.0-1.el9.noa 7.7 MB/s | 254 kB 00:00 2026-02-22T22:59:56.870 INFO:teuthology.orchestra.run.trial168.stdout:(139/173): python3-grpcio-tools-1.46.7-10.el9.x 4.4 MB/s | 144 kB 00:00 2026-02-22T22:59:56.902 INFO:teuthology.orchestra.run.trial168.stdout:(140/173): python3-influxdb-5.3.1-1.el9.noarch. 4.2 MB/s | 139 kB 00:00 2026-02-22T22:59:56.911 INFO:teuthology.orchestra.run.trial168.stdout:(141/173): python3-grpcio-1.46.7-10.el9.x86_64. 23 MB/s | 2.0 MB 00:00 2026-02-22T22:59:56.934 INFO:teuthology.orchestra.run.trial168.stdout:(142/173): python3-isodate-0.6.1-3.el9.noarch.r 1.8 MB/s | 56 kB 00:00 2026-02-22T22:59:56.940 INFO:teuthology.orchestra.run.trial168.stdout:(143/173): python3-jaraco-8.2.1-3.el9.noarch.rp 372 kB/s | 11 kB 00:00 2026-02-22T22:59:56.963 INFO:teuthology.orchestra.run.trial168.stdout:(144/173): python3-jaraco-classes-3.2.1-5.el9.n 588 kB/s | 18 kB 00:00 2026-02-22T22:59:56.968 INFO:teuthology.orchestra.run.trial168.stdout:(145/173): python3-jaraco-collections-3.0.0-8.e 805 kB/s | 23 kB 00:00 2026-02-22T22:59:56.993 INFO:teuthology.orchestra.run.trial168.stdout:(146/173): python3-jaraco-context-6.0.1-3.el9.n 655 kB/s | 20 kB 00:00 2026-02-22T22:59:56.997 INFO:teuthology.orchestra.run.trial168.stdout:(147/173): python3-jaraco-functools-3.5.0-2.el9 676 kB/s | 19 kB 00:00 2026-02-22T22:59:57.023 INFO:teuthology.orchestra.run.trial168.stdout:(148/173): python3-jaraco-text-4.0.0-2.el9.noar 877 kB/s | 26 kB 00:00 2026-02-22T22:59:57.030 INFO:teuthology.orchestra.run.trial168.stdout:(149/173): python3-kubernetes-26.1.0-3.el9.noar 31 MB/s | 1.0 MB 00:00 2026-02-22T22:59:57.054 INFO:teuthology.orchestra.run.trial168.stdout:(150/173): python3-logutils-0.3.5-21.el9.noarch 1.5 MB/s | 46 kB 00:00 2026-02-22T22:59:57.060 INFO:teuthology.orchestra.run.trial168.stdout:(151/173): python3-more-itertools-8.12.0-2.el9. 2.6 MB/s | 79 kB 00:00 2026-02-22T22:59:57.085 INFO:teuthology.orchestra.run.trial168.stdout:(152/173): python3-msgpack-1.0.3-2.el9.x86_64.r 2.7 MB/s | 86 kB 00:00 2026-02-22T22:59:57.090 INFO:teuthology.orchestra.run.trial168.stdout:(153/173): python3-natsort-7.1.1-5.el9.noarch.r 1.9 MB/s | 58 kB 00:00 2026-02-22T22:59:57.118 INFO:teuthology.orchestra.run.trial168.stdout:(154/173): python3-pecan-1.4.2-3.el9.noarch.rpm 8.1 MB/s | 272 kB 00:00 2026-02-22T22:59:57.119 INFO:teuthology.orchestra.run.trial168.stdout:(155/173): python3-portend-3.1.0-2.el9.noarch.r 557 kB/s | 16 kB 00:00 2026-02-22T22:59:57.149 INFO:teuthology.orchestra.run.trial168.stdout:(156/173): python3-repoze-lru-0.7-16.el9.noarch 1.0 MB/s | 31 kB 00:00 2026-02-22T22:59:57.149 INFO:teuthology.orchestra.run.trial168.stdout:(157/173): python3-pyOpenSSL-21.0.0-1.el9.noarc 2.8 MB/s | 90 kB 00:00 2026-02-22T22:59:57.180 INFO:teuthology.orchestra.run.trial168.stdout:(158/173): python3-routes-2.5.1-5.el9.noarch.rp 5.9 MB/s | 188 kB 00:00 2026-02-22T22:59:57.180 INFO:teuthology.orchestra.run.trial168.stdout:(159/173): python3-rsa-4.9-2.el9.noarch.rpm 1.9 MB/s | 59 kB 00:00 2026-02-22T22:59:57.211 INFO:teuthology.orchestra.run.trial168.stdout:(160/173): python3-tempora-5.0.0-2.el9.noarch.r 1.2 MB/s | 36 kB 00:00 2026-02-22T22:59:57.211 INFO:teuthology.orchestra.run.trial168.stdout:(161/173): python3-saml-1.16.0-1.el9.noarch.rpm 3.9 MB/s | 125 kB 00:00 2026-02-22T22:59:57.242 INFO:teuthology.orchestra.run.trial168.stdout:(162/173): python3-typing-extensions-4.15.0-1.e 2.7 MB/s | 86 kB 00:00 2026-02-22T22:59:57.243 INFO:teuthology.orchestra.run.trial168.stdout:(163/173): python3-webob-1.8.8-2.el9.noarch.rpm 7.2 MB/s | 230 kB 00:00 2026-02-22T22:59:57.273 INFO:teuthology.orchestra.run.trial168.stdout:(164/173): python3-websocket-client-1.2.3-2.el9 2.8 MB/s | 90 kB 00:00 2026-02-22T22:59:57.279 INFO:teuthology.orchestra.run.trial168.stdout:(165/173): python3-werkzeug-2.0.3-3.el9.1.noarc 11 MB/s | 427 kB 00:00 2026-02-22T22:59:57.304 INFO:teuthology.orchestra.run.trial168.stdout:(166/173): python3-xmlsec-1.3.13-1.el9.x86_64.r 1.5 MB/s | 48 kB 00:00 2026-02-22T22:59:57.308 INFO:teuthology.orchestra.run.trial168.stdout:(167/173): python3-xmltodict-0.12.0-15.el9.noar 766 kB/s | 22 kB 00:00 2026-02-22T22:59:57.335 INFO:teuthology.orchestra.run.trial168.stdout:(168/173): python3-zc-lockfile-2.0-10.el9.noarc 657 kB/s | 20 kB 00:00 2026-02-22T22:59:57.339 INFO:teuthology.orchestra.run.trial168.stdout:(169/173): re2-20211101-20.el9.x86_64.rpm 6.1 MB/s | 191 kB 00:00 2026-02-22T22:59:57.426 INFO:teuthology.orchestra.run.trial168.stdout:(170/173): thrift-0.15.0-4.el9.x86_64.rpm 17 MB/s | 1.6 MB 00:00 2026-02-22T22:59:58.688 INFO:teuthology.orchestra.run.trial168.stdout:(171/173): librados2-19.2.3-689.gf90f4eae.el9.x 2.5 MB/s | 3.4 MB 00:01 2026-02-22T22:59:59.844 INFO:teuthology.orchestra.run.trial168.stdout:(172/173): librbd1-19.2.3-689.gf90f4eae.el9.x86 1.3 MB/s | 3.2 MB 00:02 2026-02-22T23:00:09.301 INFO:teuthology.orchestra.run.trial168.stdout:(173/173): ceph-test-debuginfo-19.2.3-689.gf90f 27 MB/s | 729 MB 00:26 2026-02-22T23:00:09.303 INFO:teuthology.orchestra.run.trial168.stdout:-------------------------------------------------------------------------------- 2026-02-22T23:00:09.304 INFO:teuthology.orchestra.run.trial168.stdout:Total 44 MB/s | 2.3 GB 00:53 2026-02-22T23:00:09.816 INFO:teuthology.orchestra.run.trial168.stdout:Running transaction check 2026-02-22T23:00:09.871 INFO:teuthology.orchestra.run.trial168.stdout:Transaction check succeeded. 2026-02-22T23:00:09.871 INFO:teuthology.orchestra.run.trial168.stdout:Running transaction test 2026-02-22T23:00:17.031 INFO:teuthology.orchestra.run.trial168.stdout:Transaction test succeeded. 2026-02-22T23:00:17.031 INFO:teuthology.orchestra.run.trial168.stdout:Running transaction 2026-02-22T23:00:26.387 INFO:teuthology.orchestra.run.trial168.stdout: Preparing : 1/1 2026-02-22T23:00:27.648 INFO:teuthology.orchestra.run.trial168.stdout: Installing : ceph-debugsource-2:19.2.3-689.gf90f4eae.el9.x86_ 1/175 2026-02-22T23:00:27.653 INFO:teuthology.orchestra.run.trial168.stdout: Installing : ceph-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 2/175 2026-02-22T23:00:27.665 INFO:teuthology.orchestra.run.trial168.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 3/175 2026-02-22T23:00:27.678 INFO:teuthology.orchestra.run.trial168.stdout: Installing : thrift-0.15.0-4.el9.x86_64 4/175 2026-02-22T23:00:27.830 INFO:teuthology.orchestra.run.trial168.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 5/175 2026-02-22T23:00:27.833 INFO:teuthology.orchestra.run.trial168.stdout: Upgrading : librados2-2:19.2.3-689.gf90f4eae.el9.x86_64 6/175 2026-02-22T23:00:27.886 INFO:teuthology.orchestra.run.trial168.stdout: Running scriptlet: librados2-2:19.2.3-689.gf90f4eae.el9.x86_64 6/175 2026-02-22T23:00:27.886 INFO:teuthology.orchestra.run.trial168.stdout: Installing : libcephfs2-2:19.2.3-689.gf90f4eae.el9.x86_64 7/175 2026-02-22T23:00:27.911 INFO:teuthology.orchestra.run.trial168.stdout: Running scriptlet: libcephfs2-2:19.2.3-689.gf90f4eae.el9.x86_64 7/175 2026-02-22T23:00:27.917 INFO:teuthology.orchestra.run.trial168.stdout: Installing : python3-rados-2:19.2.3-689.gf90f4eae.el9.x86_64 8/175 2026-02-22T23:00:27.924 INFO:teuthology.orchestra.run.trial168.stdout: Installing : libxslt-1.1.34-12.el9.x86_64 9/175 2026-02-22T23:00:27.929 INFO:teuthology.orchestra.run.trial168.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 10/175 2026-02-22T23:00:27.930 INFO:teuthology.orchestra.run.trial168.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 11/175 2026-02-22T23:00:27.935 INFO:teuthology.orchestra.run.trial168.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 12/175 2026-02-22T23:00:27.968 INFO:teuthology.orchestra.run.trial168.stdout: Installing : libnbd-1.20.3-4.el9.x86_64 13/175 2026-02-22T23:00:27.975 INFO:teuthology.orchestra.run.trial168.stdout: Installing : python3-lxml-4.6.5-3.el9.x86_64 14/175 2026-02-22T23:00:27.982 INFO:teuthology.orchestra.run.trial168.stdout: Installing : xmlsec1-1.2.29-13.el9.x86_64 15/175 2026-02-22T23:00:27.983 INFO:teuthology.orchestra.run.trial168.stdout: Installing : libcephsqlite-2:19.2.3-689.gf90f4eae.el9.x86_64 16/175 2026-02-22T23:00:28.013 INFO:teuthology.orchestra.run.trial168.stdout: Running scriptlet: libcephsqlite-2:19.2.3-689.gf90f4eae.el9.x86_64 16/175 2026-02-22T23:00:28.014 INFO:teuthology.orchestra.run.trial168.stdout: Installing : libradosstriper1-2:19.2.3-689.gf90f4eae.el9.x86_ 17/175 2026-02-22T23:00:28.025 INFO:teuthology.orchestra.run.trial168.stdout: Running scriptlet: libradosstriper1-2:19.2.3-689.gf90f4eae.el9.x86_ 17/175 2026-02-22T23:00:28.054 INFO:teuthology.orchestra.run.trial168.stdout: Installing : re2-1:20211101-20.el9.x86_64 18/175 2026-02-22T23:00:28.085 INFO:teuthology.orchestra.run.trial168.stdout: Installing : libarrow-9.0.0-15.el9.x86_64 19/175 2026-02-22T23:00:28.089 INFO:teuthology.orchestra.run.trial168.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 20/175 2026-02-22T23:00:28.094 INFO:teuthology.orchestra.run.trial168.stdout: Installing : liboath-2.6.12-1.el9.x86_64 21/175 2026-02-22T23:00:28.097 INFO:teuthology.orchestra.run.trial168.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 22/175 2026-02-22T23:00:28.118 INFO:teuthology.orchestra.run.trial168.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 23/175 2026-02-22T23:00:28.126 INFO:teuthology.orchestra.run.trial168.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/175 2026-02-22T23:00:28.134 INFO:teuthology.orchestra.run.trial168.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 25/175 2026-02-22T23:00:28.140 INFO:teuthology.orchestra.run.trial168.stdout: Installing : protobuf-3.14.0-17.el9.x86_64 26/175 2026-02-22T23:00:28.142 INFO:teuthology.orchestra.run.trial168.stdout: Installing : lua-5.4.4-4.el9.x86_64 27/175 2026-02-22T23:00:28.146 INFO:teuthology.orchestra.run.trial168.stdout: Installing : flexiblas-3.0.4-9.el9.x86_64 28/175 2026-02-22T23:00:28.169 INFO:teuthology.orchestra.run.trial168.stdout: Installing : unzip-6.0-59.el9.x86_64 29/175 2026-02-22T23:00:28.182 INFO:teuthology.orchestra.run.trial168.stdout: Installing : python3-urllib3-1.26.5-7.el9.noarch 30/175 2026-02-22T23:00:28.185 INFO:teuthology.orchestra.run.trial168.stdout: Installing : python3-requests-2.25.1-10.el9.noarch 31/175 2026-02-22T23:00:28.191 INFO:teuthology.orchestra.run.trial168.stdout: Installing : libquadmath-11.5.0-14.el9.x86_64 32/175 2026-02-22T23:00:28.193 INFO:teuthology.orchestra.run.trial168.stdout: Installing : libgfortran-11.5.0-14.el9.x86_64 33/175 2026-02-22T23:00:28.218 INFO:teuthology.orchestra.run.trial168.stdout: Installing : ledmon-libs-1.1.0-3.el9.x86_64 34/175 2026-02-22T23:00:28.222 INFO:teuthology.orchestra.run.trial168.stdout: Installing : python3-ceph-common-2:19.2.3-689.gf90f4eae.el9.x 35/175 2026-02-22T23:00:28.231 INFO:teuthology.orchestra.run.trial168.stdout: Installing : python3-ceph-argparse-2:19.2.3-689.gf90f4eae.el9 36/175 2026-02-22T23:00:28.242 INFO:teuthology.orchestra.run.trial168.stdout: Installing : python3-cephfs-2:19.2.3-689.gf90f4eae.el9.x86_64 37/175 2026-02-22T23:00:28.248 INFO:teuthology.orchestra.run.trial168.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 38/175 2026-02-22T23:00:28.272 INFO:teuthology.orchestra.run.trial168.stdout: Installing : zip-3.0-35.el9.x86_64 39/175 2026-02-22T23:00:28.276 INFO:teuthology.orchestra.run.trial168.stdout: Installing : luarocks-3.9.2-5.el9.noarch 40/175 2026-02-22T23:00:28.283 INFO:teuthology.orchestra.run.trial168.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 41/175 2026-02-22T23:00:28.307 INFO:teuthology.orchestra.run.trial168.stdout: Installing : protobuf-compiler-3.14.0-17.el9.x86_64 42/175 2026-02-22T23:00:28.358 INFO:teuthology.orchestra.run.trial168.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 43/175 2026-02-22T23:00:28.371 INFO:teuthology.orchestra.run.trial168.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 44/175 2026-02-22T23:00:28.378 INFO:teuthology.orchestra.run.trial168.stdout: Installing : python3-rsa-4.9-2.el9.noarch 45/175 2026-02-22T23:00:28.381 INFO:teuthology.orchestra.run.trial168.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 46/175 2026-02-22T23:00:28.383 INFO:teuthology.orchestra.run.trial168.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 47/175 2026-02-22T23:00:28.388 INFO:teuthology.orchestra.run.trial168.stdout: Installing : xmlsec1-openssl-1.2.29-13.el9.x86_64 48/175 2026-02-22T23:00:28.393 INFO:teuthology.orchestra.run.trial168.stdout: Installing : python3-xmlsec-1.3.13-1.el9.x86_64 49/175 2026-02-22T23:00:28.396 INFO:teuthology.orchestra.run.trial168.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 50/175 2026-02-22T23:00:28.404 INFO:teuthology.orchestra.run.trial168.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 51/175 2026-02-22T23:00:28.409 INFO:teuthology.orchestra.run.trial168.stdout: Installing : librados-devel-2:19.2.3-689.gf90f4eae.el9.x86_64 52/175 2026-02-22T23:00:28.412 INFO:teuthology.orchestra.run.trial168.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 53/175 2026-02-22T23:00:28.426 INFO:teuthology.orchestra.run.trial168.stdout: Installing : python3-xmltodict-0.12.0-15.el9.noarch 54/175 2026-02-22T23:00:28.446 INFO:teuthology.orchestra.run.trial168.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 55/175 2026-02-22T23:00:28.451 INFO:teuthology.orchestra.run.trial168.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 56/175 2026-02-22T23:00:28.456 INFO:teuthology.orchestra.run.trial168.stdout: Installing : python3-typing-extensions-4.15.0-1.el9.noarch 57/175 2026-02-22T23:00:28.467 INFO:teuthology.orchestra.run.trial168.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 58/175 2026-02-22T23:00:28.476 INFO:teuthology.orchestra.run.trial168.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 59/175 2026-02-22T23:00:28.483 INFO:teuthology.orchestra.run.trial168.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 60/175 2026-02-22T23:00:28.502 INFO:teuthology.orchestra.run.trial168.stdout: Installing : python3-msgpack-1.0.3-2.el9.x86_64 61/175 2026-02-22T23:00:28.511 INFO:teuthology.orchestra.run.trial168.stdout: Installing : python3-influxdb-5.3.1-1.el9.noarch 62/175 2026-02-22T23:00:28.564 INFO:teuthology.orchestra.run.trial168.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 63/175 2026-02-22T23:00:28.576 INFO:teuthology.orchestra.run.trial168.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 64/175 2026-02-22T23:00:28.592 INFO:teuthology.orchestra.run.trial168.stdout: Installing : python3-isodate-0.6.1-3.el9.noarch 65/175 2026-02-22T23:00:28.647 INFO:teuthology.orchestra.run.trial168.stdout: Installing : python3-saml-1.16.0-1.el9.noarch 66/175 2026-02-22T23:00:28.647 INFO:teuthology.orchestra.run.trial168.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 67/175 2026-02-22T23:00:28.647 INFO:teuthology.orchestra.run.trial168.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 68/175 2026-02-22T23:00:28.966 INFO:teuthology.orchestra.run.trial168.stdout: Installing : python3-google-auth-1:2.45.0-1.el9.noarch 69/175 2026-02-22T23:00:28.975 INFO:teuthology.orchestra.run.trial168.stdout: Installing : python3-kubernetes-1:26.1.0-3.el9.noarch 70/175 2026-02-22T23:00:28.979 INFO:teuthology.orchestra.run.trial168.stdout: Installing : python3-backports-tarfile-1.2.0-1.el9.noarch 71/175 2026-02-22T23:00:28.985 INFO:teuthology.orchestra.run.trial168.stdout: Installing : python3-jaraco-context-6.0.1-3.el9.noarch 72/175 2026-02-22T23:00:28.991 INFO:teuthology.orchestra.run.trial168.stdout: Installing : python3-autocommand-2.2.2-8.el9.noarch 73/175 2026-02-22T23:00:28.994 INFO:teuthology.orchestra.run.trial168.stdout: Installing : python3-jaraco-text-4.0.0-2.el9.noarch 74/175 2026-02-22T23:00:28.997 INFO:teuthology.orchestra.run.trial168.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 75/175 2026-02-22T23:00:29.003 INFO:teuthology.orchestra.run.trial168.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 76/175 2026-02-22T23:00:29.005 INFO:teuthology.orchestra.run.trial168.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 77/175 2026-02-22T23:00:29.006 INFO:teuthology.orchestra.run.trial168.stdout: Installing : libarrow-doc-9.0.0-15.el9.noarch 78/175 2026-02-22T23:00:29.036 INFO:teuthology.orchestra.run.trial168.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 79/175 2026-02-22T23:00:29.081 INFO:teuthology.orchestra.run.trial168.stdout: Installing : abseil-cpp-20211102.0-4.el9.x86_64 80/175 2026-02-22T23:00:29.090 INFO:teuthology.orchestra.run.trial168.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 81/175 2026-02-22T23:00:29.136 INFO:teuthology.orchestra.run.trial168.stdout: Installing : socat-1.7.4.1-8.el9.x86_64 82/175 2026-02-22T23:00:29.165 INFO:teuthology.orchestra.run.trial168.stdout: Installing : python3-py-1.10.0-6.el9.noarch 83/175 2026-02-22T23:00:29.175 INFO:teuthology.orchestra.run.trial168.stdout: Installing : python3-protobuf-3.14.0-17.el9.noarch 84/175 2026-02-22T23:00:29.182 INFO:teuthology.orchestra.run.trial168.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 85/175 2026-02-22T23:00:29.186 INFO:teuthology.orchestra.run.trial168.stdout: Installing : python3-pluggy-0.13.1-7.el9.noarch 86/175 2026-02-22T23:00:29.220 INFO:teuthology.orchestra.run.trial168.stdout: Installing : python3-iniconfig-1.1.1-7.el9.noarch 87/175 2026-02-22T23:00:29.444 INFO:teuthology.orchestra.run.trial168.stdout: Installing : python3-devel-3.9.25-3.el9.x86_64 88/175 2026-02-22T23:00:29.468 INFO:teuthology.orchestra.run.trial168.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 89/175 2026-02-22T23:00:29.472 INFO:teuthology.orchestra.run.trial168.stdout: Installing : python3-jinja2-2.11.3-8.el9.noarch 90/175 2026-02-22T23:00:29.530 INFO:teuthology.orchestra.run.trial168.stdout: Installing : openblas-0.3.29-1.el9.x86_64 91/175 2026-02-22T23:00:29.533 INFO:teuthology.orchestra.run.trial168.stdout: Installing : openblas-openmp-0.3.29-1.el9.x86_64 92/175 2026-02-22T23:00:29.554 INFO:teuthology.orchestra.run.trial168.stdout: Installing : flexiblas-openblas-openmp-3.0.4-9.el9.x86_64 93/175 2026-02-22T23:00:29.880 INFO:teuthology.orchestra.run.trial168.stdout: Installing : flexiblas-netlib-3.0.4-9.el9.x86_64 94/175 2026-02-22T23:00:29.954 INFO:teuthology.orchestra.run.trial168.stdout: Installing : python3-numpy-1:1.23.5-2.el9.x86_64 95/175 2026-02-22T23:00:30.627 INFO:teuthology.orchestra.run.trial168.stdout: Installing : python3-numpy-f2py-1:1.23.5-2.el9.x86_64 96/175 2026-02-22T23:00:30.643 INFO:teuthology.orchestra.run.trial168.stdout: Installing : python3-scipy-1.9.3-2.el9.x86_64 97/175 2026-02-22T23:00:30.782 INFO:teuthology.orchestra.run.trial168.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 98/175 2026-02-22T23:00:30.783 INFO:teuthology.orchestra.run.trial168.stdout: Upgrading : librbd1-2:19.2.3-689.gf90f4eae.el9.x86_64 99/175 2026-02-22T23:00:30.809 INFO:teuthology.orchestra.run.trial168.stdout: Running scriptlet: librbd1-2:19.2.3-689.gf90f4eae.el9.x86_64 99/175 2026-02-22T23:00:30.811 INFO:teuthology.orchestra.run.trial168.stdout: Installing : python3-rbd-2:19.2.3-689.gf90f4eae.el9.x86_64 100/175 2026-02-22T23:00:30.817 INFO:teuthology.orchestra.run.trial168.stdout: Installing : boost-program-options-1.75.0-13.el9.x86_64 101/175 2026-02-22T23:00:31.040 INFO:teuthology.orchestra.run.trial168.stdout: Installing : parquet-libs-9.0.0-15.el9.x86_64 102/175 2026-02-22T23:00:31.041 INFO:teuthology.orchestra.run.trial168.stdout: Installing : librgw2-2:19.2.3-689.gf90f4eae.el9.x86_64 103/175 2026-02-22T23:00:31.056 INFO:teuthology.orchestra.run.trial168.stdout: Running scriptlet: librgw2-2:19.2.3-689.gf90f4eae.el9.x86_64 103/175 2026-02-22T23:00:31.057 INFO:teuthology.orchestra.run.trial168.stdout: Installing : python3-rgw-2:19.2.3-689.gf90f4eae.el9.x86_64 104/175 2026-02-22T23:00:32.028 INFO:teuthology.orchestra.run.trial168.stdout: Running scriptlet: ceph-common-2:19.2.3-689.gf90f4eae.el9.x86_64 105/175 2026-02-22T23:00:32.030 INFO:teuthology.orchestra.run.trial168.stdout: Installing : ceph-common-2:19.2.3-689.gf90f4eae.el9.x86_64 105/175 2026-02-22T23:00:32.046 INFO:teuthology.orchestra.run.trial168.stdout: Running scriptlet: ceph-common-2:19.2.3-689.gf90f4eae.el9.x86_64 105/175 2026-02-22T23:00:32.056 INFO:teuthology.orchestra.run.trial168.stdout: Installing : python3-pyparsing-2.4.7-9.el9.noarch 106/175 2026-02-22T23:00:32.062 INFO:teuthology.orchestra.run.trial168.stdout: Installing : python3-packaging-20.9-5.el9.noarch 107/175 2026-02-22T23:00:32.076 INFO:teuthology.orchestra.run.trial168.stdout: Installing : python3-ply-3.11-14.el9.noarch 108/175 2026-02-22T23:00:32.091 INFO:teuthology.orchestra.run.trial168.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 109/175 2026-02-22T23:00:32.167 INFO:teuthology.orchestra.run.trial168.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 110/175 2026-02-22T23:00:32.176 INFO:teuthology.orchestra.run.trial168.stdout: Installing : python3-cryptography-36.0.1-5.el9.x86_64 111/175 2026-02-22T23:00:32.199 INFO:teuthology.orchestra.run.trial168.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 112/175 2026-02-22T23:00:32.228 INFO:teuthology.orchestra.run.trial168.stdout: Installing : python3-cheroot-10.0.1-4.el9.noarch 113/175 2026-02-22T23:00:32.279 INFO:teuthology.orchestra.run.trial168.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 114/175 2026-02-22T23:00:32.285 INFO:teuthology.orchestra.run.trial168.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 115/175 2026-02-22T23:00:32.289 INFO:teuthology.orchestra.run.trial168.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 116/175 2026-02-22T23:00:32.293 INFO:teuthology.orchestra.run.trial168.stdout: Installing : pciutils-3.7.0-7.el9.x86_64 117/175 2026-02-22T23:00:32.295 INFO:teuthology.orchestra.run.trial168.stdout: Installing : qatlib-25.08.0-2.el9.x86_64 118/175 2026-02-22T23:00:32.296 INFO:teuthology.orchestra.run.trial168.stdout: Installing : qatlib-service-25.08.0-2.el9.x86_64 119/175 2026-02-22T23:00:32.312 INFO:teuthology.orchestra.run.trial168.stdout: Running scriptlet: qatlib-service-25.08.0-2.el9.x86_64 119/175 2026-02-22T23:00:32.427 INFO:teuthology.orchestra.run.trial168.stdout: Installing : qatzip-libs-1.3.1-1.el9.x86_64 120/175 2026-02-22T23:00:32.429 INFO:teuthology.orchestra.run.trial168.stdout: Installing : nvme-cli-2.16-1.el9.x86_64 121/175 2026-02-22T23:00:32.709 INFO:teuthology.orchestra.run.trial168.stdout: Running scriptlet: nvme-cli-2.16-1.el9.x86_64 121/175 2026-02-22T23:00:32.709 INFO:teuthology.orchestra.run.trial168.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service → /usr/lib/systemd/system/nvmefc-boot-connections.service. 2026-02-22T23:00:32.709 INFO:teuthology.orchestra.run.trial168.stdout: 2026-02-22T23:00:32.986 INFO:teuthology.orchestra.run.trial168.stdout: Installing : ceph-base-2:19.2.3-689.gf90f4eae.el9.x86_64 122/175 2026-02-22T23:00:33.023 INFO:teuthology.orchestra.run.trial168.stdout: Running scriptlet: ceph-base-2:19.2.3-689.gf90f4eae.el9.x86_64 122/175 2026-02-22T23:00:33.023 INFO:teuthology.orchestra.run.trial168.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2026-02-22T23:00:33.023 INFO:teuthology.orchestra.run.trial168.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2026-02-22T23:00:33.023 INFO:teuthology.orchestra.run.trial168.stdout: 2026-02-22T23:00:33.026 INFO:teuthology.orchestra.run.trial168.stdout: Installing : ceph-selinux-2:19.2.3-689.gf90f4eae.el9.x86_64 123/175 2026-02-22T23:00:37.984 INFO:teuthology.orchestra.run.trial168.stdout: Running scriptlet: ceph-selinux-2:19.2.3-689.gf90f4eae.el9.x86_64 123/175 2026-02-22T23:00:37.984 INFO:teuthology.orchestra.run.trial168.stdout:skipping the directory /sys 2026-02-22T23:00:37.984 INFO:teuthology.orchestra.run.trial168.stdout:skipping the directory /proc 2026-02-22T23:00:37.985 INFO:teuthology.orchestra.run.trial168.stdout:skipping the directory /mnt 2026-02-22T23:00:37.985 INFO:teuthology.orchestra.run.trial168.stdout:skipping the directory /var/tmp 2026-02-22T23:00:37.985 INFO:teuthology.orchestra.run.trial168.stdout:skipping the directory /home 2026-02-22T23:00:37.985 INFO:teuthology.orchestra.run.trial168.stdout:skipping the directory /root 2026-02-22T23:00:37.985 INFO:teuthology.orchestra.run.trial168.stdout:skipping the directory /tmp 2026-02-22T23:00:37.985 INFO:teuthology.orchestra.run.trial168.stdout: 2026-02-22T23:00:38.094 INFO:teuthology.orchestra.run.trial168.stdout: Installing : ceph-mds-2:19.2.3-689.gf90f4eae.el9.x86_64 124/175 2026-02-22T23:00:38.116 INFO:teuthology.orchestra.run.trial168.stdout: Running scriptlet: ceph-mds-2:19.2.3-689.gf90f4eae.el9.x86_64 124/175 2026-02-22T23:00:38.116 INFO:teuthology.orchestra.run.trial168.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-22T23:00:38.116 INFO:teuthology.orchestra.run.trial168.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2026-02-22T23:00:38.116 INFO:teuthology.orchestra.run.trial168.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2026-02-22T23:00:38.116 INFO:teuthology.orchestra.run.trial168.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2026-02-22T23:00:38.116 INFO:teuthology.orchestra.run.trial168.stdout: 2026-02-22T23:00:38.322 INFO:teuthology.orchestra.run.trial168.stdout: Installing : ceph-mon-2:19.2.3-689.gf90f4eae.el9.x86_64 125/175 2026-02-22T23:00:38.343 INFO:teuthology.orchestra.run.trial168.stdout: Running scriptlet: ceph-mon-2:19.2.3-689.gf90f4eae.el9.x86_64 125/175 2026-02-22T23:00:38.343 INFO:teuthology.orchestra.run.trial168.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-22T23:00:38.343 INFO:teuthology.orchestra.run.trial168.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2026-02-22T23:00:38.343 INFO:teuthology.orchestra.run.trial168.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2026-02-22T23:00:38.343 INFO:teuthology.orchestra.run.trial168.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2026-02-22T23:00:38.343 INFO:teuthology.orchestra.run.trial168.stdout: 2026-02-22T23:00:38.349 INFO:teuthology.orchestra.run.trial168.stdout: Installing : mailcap-2.1.49-5.el9.noarch 126/175 2026-02-22T23:00:38.350 INFO:teuthology.orchestra.run.trial168.stdout: Installing : libconfig-1.7.2-9.el9.x86_64 127/175 2026-02-22T23:00:38.368 INFO:teuthology.orchestra.run.trial168.stdout: Running scriptlet: libstoragemgmt-1.10.1-1.el9.x86_64 128/175 2026-02-22T23:00:38.368 INFO:teuthology.orchestra.run.trial168.stdout:Creating group 'qat' with GID 992. 2026-02-22T23:00:38.368 INFO:teuthology.orchestra.run.trial168.stdout:Creating group 'libstoragemgmt' with GID 991. 2026-02-22T23:00:38.368 INFO:teuthology.orchestra.run.trial168.stdout:Creating user 'libstoragemgmt' (daemon account for libstoragemgmt) with UID 991 and GID 991. 2026-02-22T23:00:38.368 INFO:teuthology.orchestra.run.trial168.stdout: 2026-02-22T23:00:38.376 INFO:teuthology.orchestra.run.trial168.stdout: Installing : libstoragemgmt-1.10.1-1.el9.x86_64 128/175 2026-02-22T23:00:38.400 INFO:teuthology.orchestra.run.trial168.stdout: Running scriptlet: libstoragemgmt-1.10.1-1.el9.x86_64 128/175 2026-02-22T23:00:38.400 INFO:teuthology.orchestra.run.trial168.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/libstoragemgmt.service → /usr/lib/systemd/system/libstoragemgmt.service. 2026-02-22T23:00:38.400 INFO:teuthology.orchestra.run.trial168.stdout: 2026-02-22T23:00:38.434 INFO:teuthology.orchestra.run.trial168.stdout: Installing : python3-libstoragemgmt-1.10.1-1.el9.x86_64 129/175 2026-02-22T23:00:38.497 INFO:teuthology.orchestra.run.trial168.stdout: Installing : cryptsetup-2.8.1-3.el9.x86_64 130/175 2026-02-22T23:00:38.499 INFO:teuthology.orchestra.run.trial168.stdout: Installing : ceph-volume-2:19.2.3-689.gf90f4eae.el9.noarch 131/175 2026-02-22T23:00:38.511 INFO:teuthology.orchestra.run.trial168.stdout: Running scriptlet: ceph-volume-2:19.2.3-689.gf90f4eae.el9.noarch 131/175 2026-02-22T23:00:38.512 INFO:teuthology.orchestra.run.trial168.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-22T23:00:38.512 INFO:teuthology.orchestra.run.trial168.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2026-02-22T23:00:38.512 INFO:teuthology.orchestra.run.trial168.stdout: 2026-02-22T23:00:39.206 INFO:teuthology.orchestra.run.trial168.stdout: Installing : ceph-osd-2:19.2.3-689.gf90f4eae.el9.x86_64 132/175 2026-02-22T23:00:39.230 INFO:teuthology.orchestra.run.trial168.stdout: Running scriptlet: ceph-osd-2:19.2.3-689.gf90f4eae.el9.x86_64 132/175 2026-02-22T23:00:39.230 INFO:teuthology.orchestra.run.trial168.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-22T23:00:39.230 INFO:teuthology.orchestra.run.trial168.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2026-02-22T23:00:39.230 INFO:teuthology.orchestra.run.trial168.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2026-02-22T23:00:39.230 INFO:teuthology.orchestra.run.trial168.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2026-02-22T23:00:39.230 INFO:teuthology.orchestra.run.trial168.stdout: 2026-02-22T23:00:39.287 INFO:teuthology.orchestra.run.trial168.stdout: Running scriptlet: cephadm-2:19.2.3-689.gf90f4eae.el9.noarch 133/175 2026-02-22T23:00:39.289 INFO:teuthology.orchestra.run.trial168.stdout: Installing : cephadm-2:19.2.3-689.gf90f4eae.el9.noarch 133/175 2026-02-22T23:00:39.293 INFO:teuthology.orchestra.run.trial168.stdout: Installing : ceph-prometheus-alerts-2:19.2.3-689.gf90f4eae.el 134/175 2026-02-22T23:00:39.313 INFO:teuthology.orchestra.run.trial168.stdout: Installing : ceph-grafana-dashboards-2:19.2.3-689.gf90f4eae.e 135/175 2026-02-22T23:00:39.314 INFO:teuthology.orchestra.run.trial168.stdout: Installing : ceph-mgr-cephadm-2:19.2.3-689.gf90f4eae.el9.noar 136/175 2026-02-22T23:00:39.799 INFO:teuthology.orchestra.run.trial168.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.2.3-689.gf90f4eae.el9.noar 136/175 2026-02-22T23:00:39.802 INFO:teuthology.orchestra.run.trial168.stdout: Installing : ceph-mgr-dashboard-2:19.2.3-689.gf90f4eae.el9.no 137/175 2026-02-22T23:00:40.252 INFO:teuthology.orchestra.run.trial168.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.2.3-689.gf90f4eae.el9.no 137/175 2026-02-22T23:00:40.253 INFO:teuthology.orchestra.run.trial168.stdout: Installing : ceph-mgr-diskprediction-local-2:19.2.3-689.gf90f 138/175 2026-02-22T23:00:40.264 INFO:teuthology.orchestra.run.trial168.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.2.3-689.gf90f 138/175 2026-02-22T23:00:40.264 INFO:teuthology.orchestra.run.trial168.stdout: Installing : ceph-mgr-k8sevents-2:19.2.3-689.gf90f4eae.el9.no 139/175 2026-02-22T23:00:40.319 INFO:teuthology.orchestra.run.trial168.stdout: Running scriptlet: ceph-mgr-k8sevents-2:19.2.3-689.gf90f4eae.el9.no 139/175 2026-02-22T23:00:40.368 INFO:teuthology.orchestra.run.trial168.stdout: Installing : ceph-mgr-modules-core-2:19.2.3-689.gf90f4eae.el9 140/175 2026-02-22T23:00:40.369 INFO:teuthology.orchestra.run.trial168.stdout: Installing : ceph-mgr-2:19.2.3-689.gf90f4eae.el9.x86_64 141/175 2026-02-22T23:00:40.389 INFO:teuthology.orchestra.run.trial168.stdout: Running scriptlet: ceph-mgr-2:19.2.3-689.gf90f4eae.el9.x86_64 141/175 2026-02-22T23:00:40.389 INFO:teuthology.orchestra.run.trial168.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-22T23:00:40.389 INFO:teuthology.orchestra.run.trial168.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2026-02-22T23:00:40.389 INFO:teuthology.orchestra.run.trial168.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2026-02-22T23:00:40.389 INFO:teuthology.orchestra.run.trial168.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2026-02-22T23:00:40.389 INFO:teuthology.orchestra.run.trial168.stdout: 2026-02-22T23:00:40.400 INFO:teuthology.orchestra.run.trial168.stdout: Installing : ceph-mgr-rook-2:19.2.3-689.gf90f4eae.el9.noarch 142/175 2026-02-22T23:00:40.410 INFO:teuthology.orchestra.run.trial168.stdout: Running scriptlet: ceph-mgr-rook-2:19.2.3-689.gf90f4eae.el9.noarch 142/175 2026-02-22T23:00:40.855 INFO:teuthology.orchestra.run.trial168.stdout: Installing : ceph-2:19.2.3-689.gf90f4eae.el9.x86_64 143/175 2026-02-22T23:00:40.857 INFO:teuthology.orchestra.run.trial168.stdout: Installing : ceph-radosgw-2:19.2.3-689.gf90f4eae.el9.x86_64 144/175 2026-02-22T23:00:40.878 INFO:teuthology.orchestra.run.trial168.stdout: Running scriptlet: ceph-radosgw-2:19.2.3-689.gf90f4eae.el9.x86_64 144/175 2026-02-22T23:00:40.878 INFO:teuthology.orchestra.run.trial168.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-22T23:00:40.878 INFO:teuthology.orchestra.run.trial168.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2026-02-22T23:00:40.878 INFO:teuthology.orchestra.run.trial168.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2026-02-22T23:00:40.878 INFO:teuthology.orchestra.run.trial168.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2026-02-22T23:00:40.879 INFO:teuthology.orchestra.run.trial168.stdout: 2026-02-22T23:00:40.887 INFO:teuthology.orchestra.run.trial168.stdout: Installing : ceph-immutable-object-cache-2:19.2.3-689.gf90f4e 145/175 2026-02-22T23:00:40.906 INFO:teuthology.orchestra.run.trial168.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.2.3-689.gf90f4e 145/175 2026-02-22T23:00:40.906 INFO:teuthology.orchestra.run.trial168.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-22T23:00:40.906 INFO:teuthology.orchestra.run.trial168.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2026-02-22T23:00:40.906 INFO:teuthology.orchestra.run.trial168.stdout: 2026-02-22T23:00:41.043 INFO:teuthology.orchestra.run.trial168.stdout: Installing : rbd-mirror-2:19.2.3-689.gf90f4eae.el9.x86_64 146/175 2026-02-22T23:00:41.063 INFO:teuthology.orchestra.run.trial168.stdout: Running scriptlet: rbd-mirror-2:19.2.3-689.gf90f4eae.el9.x86_64 146/175 2026-02-22T23:00:41.063 INFO:teuthology.orchestra.run.trial168.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-22T23:00:41.063 INFO:teuthology.orchestra.run.trial168.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2026-02-22T23:00:41.064 INFO:teuthology.orchestra.run.trial168.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2026-02-22T23:00:41.064 INFO:teuthology.orchestra.run.trial168.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2026-02-22T23:00:41.064 INFO:teuthology.orchestra.run.trial168.stdout: 2026-02-22T23:00:43.346 INFO:teuthology.orchestra.run.trial168.stdout: Installing : python3-pytest-6.2.2-7.el9.noarch 147/175 2026-02-22T23:00:43.354 INFO:teuthology.orchestra.run.trial168.stdout: Installing : ceph-test-2:19.2.3-689.gf90f4eae.el9.x86_64 148/175 2026-02-22T23:00:43.362 INFO:teuthology.orchestra.run.trial168.stdout: Installing : rbd-fuse-2:19.2.3-689.gf90f4eae.el9.x86_64 149/175 2026-02-22T23:00:43.366 INFO:teuthology.orchestra.run.trial168.stdout: Installing : rbd-nbd-2:19.2.3-689.gf90f4eae.el9.x86_64 150/175 2026-02-22T23:00:43.416 INFO:teuthology.orchestra.run.trial168.stdout: Installing : libcephfs-devel-2:19.2.3-689.gf90f4eae.el9.x86_6 151/175 2026-02-22T23:00:47.123 INFO:teuthology.orchestra.run.trial168.stdout: Installing : ceph-fuse-2:19.2.3-689.gf90f4eae.el9.x86_64 152/175 2026-02-22T23:01:09.362 INFO:teuthology.orchestra.run.trial168.stdout: Installing : ceph-base-debuginfo-2:19.2.3-689.gf90f4eae.el9.x 153/175 2026-02-22T23:01:09.702 INFO:teuthology.orchestra.run.trial168.stdout: Installing : ceph-common-debuginfo-2:19.2.3-689.gf90f4eae.el9 154/175 2026-02-22T23:01:09.778 INFO:teuthology.orchestra.run.trial168.stdout: Installing : ceph-fuse-debuginfo-2:19.2.3-689.gf90f4eae.el9.x 155/175 2026-02-22T23:01:12.269 INFO:teuthology.orchestra.run.trial168.stdout: Installing : ceph-immutable-object-cache-debuginfo-2:19.2.3-6 156/175 2026-02-22T23:01:13.200 INFO:teuthology.orchestra.run.trial168.stdout: Installing : ceph-mds-debuginfo-2:19.2.3-689.gf90f4eae.el9.x8 157/175 2026-02-22T23:01:18.105 INFO:teuthology.orchestra.run.trial168.stdout: Installing : ceph-mgr-debuginfo-2:19.2.3-689.gf90f4eae.el9.x8 158/175 2026-02-22T23:01:32.273 INFO:teuthology.orchestra.run.trial168.stdout: Installing : ceph-mon-debuginfo-2:19.2.3-689.gf90f4eae.el9.x8 159/175 2026-02-22T23:01:41.736 INFO:teuthology.orchestra.run.trial168.stdout: Installing : ceph-osd-debuginfo-2:19.2.3-689.gf90f4eae.el9.x8 160/175 2026-02-22T23:02:17.517 INFO:teuthology.orchestra.run.trial168.stdout: Installing : ceph-radosgw-debuginfo-2:19.2.3-689.gf90f4eae.el 161/175 2026-02-22T23:02:17.814 INFO:teuthology.orchestra.run.trial168.stdout: Installing : ceph-test-debuginfo-2:19.2.3-689.gf90f4eae.el9.x 162/175 2026-02-22T23:02:17.821 INFO:teuthology.orchestra.run.trial168.stdout: Installing : libcephfs2-debuginfo-2:19.2.3-689.gf90f4eae.el9. 163/175 2026-02-22T23:02:20.303 INFO:teuthology.orchestra.run.trial168.stdout: Installing : librados-devel-debuginfo-2:19.2.3-689.gf90f4eae. 164/175 2026-02-22T23:02:23.264 INFO:teuthology.orchestra.run.trial168.stdout: Installing : librados2-debuginfo-2:19.2.3-689.gf90f4eae.el9.x 165/175 2026-02-22T23:02:23.294 INFO:teuthology.orchestra.run.trial168.stdout: Installing : librbd1-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86 166/175 2026-02-22T23:02:23.329 INFO:teuthology.orchestra.run.trial168.stdout: Installing : python3-cephfs-debuginfo-2:19.2.3-689.gf90f4eae. 167/175 2026-02-22T23:02:23.383 INFO:teuthology.orchestra.run.trial168.stdout: Installing : python3-rados-debuginfo-2:19.2.3-689.gf90f4eae.e 168/175 2026-02-22T23:02:23.398 INFO:teuthology.orchestra.run.trial168.stdout: Installing : python3-rbd-debuginfo-2:19.2.3-689.gf90f4eae.el9 169/175 2026-02-22T23:02:23.426 INFO:teuthology.orchestra.run.trial168.stdout: Installing : python3-rgw-debuginfo-2:19.2.3-689.gf90f4eae.el9 170/175 2026-02-22T23:02:26.295 INFO:teuthology.orchestra.run.trial168.stdout: Installing : rbd-fuse-debuginfo-2:19.2.3-689.gf90f4eae.el9.x8 171/175 2026-02-22T23:02:26.362 INFO:teuthology.orchestra.run.trial168.stdout: Installing : rbd-mirror-debuginfo-2:19.2.3-689.gf90f4eae.el9. 172/175 2026-02-22T23:02:26.364 INFO:teuthology.orchestra.run.trial168.stdout: Installing : rbd-nbd-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86 173/175 2026-02-22T23:02:26.365 INFO:teuthology.orchestra.run.trial168.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 174/175 2026-02-22T23:02:26.377 INFO:teuthology.orchestra.run.trial168.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 174/175 2026-02-22T23:02:26.377 INFO:teuthology.orchestra.run.trial168.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 175/175 2026-02-22T23:02:29.830 INFO:teuthology.orchestra.run.trial168.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 175/175 2026-02-22T23:02:29.830 INFO:teuthology.orchestra.run.trial168.stdout: Verifying : ceph-2:19.2.3-689.gf90f4eae.el9.x86_64 1/175 2026-02-22T23:02:29.830 INFO:teuthology.orchestra.run.trial168.stdout: Verifying : ceph-base-2:19.2.3-689.gf90f4eae.el9.x86_64 2/175 2026-02-22T23:02:29.830 INFO:teuthology.orchestra.run.trial168.stdout: Verifying : ceph-base-debuginfo-2:19.2.3-689.gf90f4eae.el9.x 3/175 2026-02-22T23:02:29.830 INFO:teuthology.orchestra.run.trial168.stdout: Verifying : ceph-common-2:19.2.3-689.gf90f4eae.el9.x86_64 4/175 2026-02-22T23:02:29.830 INFO:teuthology.orchestra.run.trial168.stdout: Verifying : ceph-common-debuginfo-2:19.2.3-689.gf90f4eae.el9 5/175 2026-02-22T23:02:29.830 INFO:teuthology.orchestra.run.trial168.stdout: Verifying : ceph-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 6/175 2026-02-22T23:02:29.830 INFO:teuthology.orchestra.run.trial168.stdout: Verifying : ceph-debugsource-2:19.2.3-689.gf90f4eae.el9.x86_ 7/175 2026-02-22T23:02:29.830 INFO:teuthology.orchestra.run.trial168.stdout: Verifying : ceph-fuse-2:19.2.3-689.gf90f4eae.el9.x86_64 8/175 2026-02-22T23:02:29.831 INFO:teuthology.orchestra.run.trial168.stdout: Verifying : ceph-fuse-debuginfo-2:19.2.3-689.gf90f4eae.el9.x 9/175 2026-02-22T23:02:29.831 INFO:teuthology.orchestra.run.trial168.stdout: Verifying : ceph-immutable-object-cache-2:19.2.3-689.gf90f4e 10/175 2026-02-22T23:02:29.831 INFO:teuthology.orchestra.run.trial168.stdout: Verifying : ceph-immutable-object-cache-debuginfo-2:19.2.3-6 11/175 2026-02-22T23:02:29.831 INFO:teuthology.orchestra.run.trial168.stdout: Verifying : ceph-mds-2:19.2.3-689.gf90f4eae.el9.x86_64 12/175 2026-02-22T23:02:29.831 INFO:teuthology.orchestra.run.trial168.stdout: Verifying : ceph-mds-debuginfo-2:19.2.3-689.gf90f4eae.el9.x8 13/175 2026-02-22T23:02:29.831 INFO:teuthology.orchestra.run.trial168.stdout: Verifying : ceph-mgr-2:19.2.3-689.gf90f4eae.el9.x86_64 14/175 2026-02-22T23:02:29.831 INFO:teuthology.orchestra.run.trial168.stdout: Verifying : ceph-mgr-debuginfo-2:19.2.3-689.gf90f4eae.el9.x8 15/175 2026-02-22T23:02:29.831 INFO:teuthology.orchestra.run.trial168.stdout: Verifying : ceph-mon-2:19.2.3-689.gf90f4eae.el9.x86_64 16/175 2026-02-22T23:02:29.831 INFO:teuthology.orchestra.run.trial168.stdout: Verifying : ceph-mon-debuginfo-2:19.2.3-689.gf90f4eae.el9.x8 17/175 2026-02-22T23:02:29.831 INFO:teuthology.orchestra.run.trial168.stdout: Verifying : ceph-osd-2:19.2.3-689.gf90f4eae.el9.x86_64 18/175 2026-02-22T23:02:29.831 INFO:teuthology.orchestra.run.trial168.stdout: Verifying : ceph-osd-debuginfo-2:19.2.3-689.gf90f4eae.el9.x8 19/175 2026-02-22T23:02:29.831 INFO:teuthology.orchestra.run.trial168.stdout: Verifying : ceph-radosgw-2:19.2.3-689.gf90f4eae.el9.x86_64 20/175 2026-02-22T23:02:29.831 INFO:teuthology.orchestra.run.trial168.stdout: Verifying : ceph-radosgw-debuginfo-2:19.2.3-689.gf90f4eae.el 21/175 2026-02-22T23:02:29.831 INFO:teuthology.orchestra.run.trial168.stdout: Verifying : ceph-selinux-2:19.2.3-689.gf90f4eae.el9.x86_64 22/175 2026-02-22T23:02:29.831 INFO:teuthology.orchestra.run.trial168.stdout: Verifying : ceph-test-2:19.2.3-689.gf90f4eae.el9.x86_64 23/175 2026-02-22T23:02:29.831 INFO:teuthology.orchestra.run.trial168.stdout: Verifying : ceph-test-debuginfo-2:19.2.3-689.gf90f4eae.el9.x 24/175 2026-02-22T23:02:29.831 INFO:teuthology.orchestra.run.trial168.stdout: Verifying : libcephfs-devel-2:19.2.3-689.gf90f4eae.el9.x86_6 25/175 2026-02-22T23:02:29.831 INFO:teuthology.orchestra.run.trial168.stdout: Verifying : libcephfs2-2:19.2.3-689.gf90f4eae.el9.x86_64 26/175 2026-02-22T23:02:29.835 INFO:teuthology.orchestra.run.trial168.stdout: Verifying : libcephfs2-debuginfo-2:19.2.3-689.gf90f4eae.el9. 27/175 2026-02-22T23:02:29.835 INFO:teuthology.orchestra.run.trial168.stdout: Verifying : libcephsqlite-2:19.2.3-689.gf90f4eae.el9.x86_64 28/175 2026-02-22T23:02:29.835 INFO:teuthology.orchestra.run.trial168.stdout: Verifying : librados-devel-2:19.2.3-689.gf90f4eae.el9.x86_64 29/175 2026-02-22T23:02:29.835 INFO:teuthology.orchestra.run.trial168.stdout: Verifying : librados-devel-debuginfo-2:19.2.3-689.gf90f4eae. 30/175 2026-02-22T23:02:29.835 INFO:teuthology.orchestra.run.trial168.stdout: Verifying : librados2-debuginfo-2:19.2.3-689.gf90f4eae.el9.x 31/175 2026-02-22T23:02:29.835 INFO:teuthology.orchestra.run.trial168.stdout: Verifying : libradosstriper1-2:19.2.3-689.gf90f4eae.el9.x86_ 32/175 2026-02-22T23:02:29.835 INFO:teuthology.orchestra.run.trial168.stdout: Verifying : librbd1-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86 33/175 2026-02-22T23:02:29.835 INFO:teuthology.orchestra.run.trial168.stdout: Verifying : librgw2-2:19.2.3-689.gf90f4eae.el9.x86_64 34/175 2026-02-22T23:02:29.835 INFO:teuthology.orchestra.run.trial168.stdout: Verifying : python3-ceph-argparse-2:19.2.3-689.gf90f4eae.el9 35/175 2026-02-22T23:02:29.835 INFO:teuthology.orchestra.run.trial168.stdout: Verifying : python3-ceph-common-2:19.2.3-689.gf90f4eae.el9.x 36/175 2026-02-22T23:02:29.835 INFO:teuthology.orchestra.run.trial168.stdout: Verifying : python3-cephfs-2:19.2.3-689.gf90f4eae.el9.x86_64 37/175 2026-02-22T23:02:29.835 INFO:teuthology.orchestra.run.trial168.stdout: Verifying : python3-cephfs-debuginfo-2:19.2.3-689.gf90f4eae. 38/175 2026-02-22T23:02:29.835 INFO:teuthology.orchestra.run.trial168.stdout: Verifying : python3-rados-2:19.2.3-689.gf90f4eae.el9.x86_64 39/175 2026-02-22T23:02:29.835 INFO:teuthology.orchestra.run.trial168.stdout: Verifying : python3-rados-debuginfo-2:19.2.3-689.gf90f4eae.e 40/175 2026-02-22T23:02:29.835 INFO:teuthology.orchestra.run.trial168.stdout: Verifying : python3-rbd-2:19.2.3-689.gf90f4eae.el9.x86_64 41/175 2026-02-22T23:02:29.835 INFO:teuthology.orchestra.run.trial168.stdout: Verifying : python3-rbd-debuginfo-2:19.2.3-689.gf90f4eae.el9 42/175 2026-02-22T23:02:29.835 INFO:teuthology.orchestra.run.trial168.stdout: Verifying : python3-rgw-2:19.2.3-689.gf90f4eae.el9.x86_64 43/175 2026-02-22T23:02:29.835 INFO:teuthology.orchestra.run.trial168.stdout: Verifying : python3-rgw-debuginfo-2:19.2.3-689.gf90f4eae.el9 44/175 2026-02-22T23:02:29.835 INFO:teuthology.orchestra.run.trial168.stdout: Verifying : rbd-fuse-2:19.2.3-689.gf90f4eae.el9.x86_64 45/175 2026-02-22T23:02:29.835 INFO:teuthology.orchestra.run.trial168.stdout: Verifying : rbd-fuse-debuginfo-2:19.2.3-689.gf90f4eae.el9.x8 46/175 2026-02-22T23:02:29.835 INFO:teuthology.orchestra.run.trial168.stdout: Verifying : rbd-mirror-2:19.2.3-689.gf90f4eae.el9.x86_64 47/175 2026-02-22T23:02:29.835 INFO:teuthology.orchestra.run.trial168.stdout: Verifying : rbd-mirror-debuginfo-2:19.2.3-689.gf90f4eae.el9. 48/175 2026-02-22T23:02:29.835 INFO:teuthology.orchestra.run.trial168.stdout: Verifying : rbd-nbd-2:19.2.3-689.gf90f4eae.el9.x86_64 49/175 2026-02-22T23:02:29.836 INFO:teuthology.orchestra.run.trial168.stdout: Verifying : rbd-nbd-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86 50/175 2026-02-22T23:02:29.836 INFO:teuthology.orchestra.run.trial168.stdout: Verifying : ceph-grafana-dashboards-2:19.2.3-689.gf90f4eae.e 51/175 2026-02-22T23:02:29.836 INFO:teuthology.orchestra.run.trial168.stdout: Verifying : ceph-mgr-cephadm-2:19.2.3-689.gf90f4eae.el9.noar 52/175 2026-02-22T23:02:29.836 INFO:teuthology.orchestra.run.trial168.stdout: Verifying : ceph-mgr-dashboard-2:19.2.3-689.gf90f4eae.el9.no 53/175 2026-02-22T23:02:29.836 INFO:teuthology.orchestra.run.trial168.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.2.3-689.gf90f 54/175 2026-02-22T23:02:29.836 INFO:teuthology.orchestra.run.trial168.stdout: Verifying : ceph-mgr-k8sevents-2:19.2.3-689.gf90f4eae.el9.no 55/175 2026-02-22T23:02:29.836 INFO:teuthology.orchestra.run.trial168.stdout: Verifying : ceph-mgr-modules-core-2:19.2.3-689.gf90f4eae.el9 56/175 2026-02-22T23:02:29.836 INFO:teuthology.orchestra.run.trial168.stdout: Verifying : ceph-mgr-rook-2:19.2.3-689.gf90f4eae.el9.noarch 57/175 2026-02-22T23:02:29.836 INFO:teuthology.orchestra.run.trial168.stdout: Verifying : ceph-prometheus-alerts-2:19.2.3-689.gf90f4eae.el 58/175 2026-02-22T23:02:29.836 INFO:teuthology.orchestra.run.trial168.stdout: Verifying : ceph-volume-2:19.2.3-689.gf90f4eae.el9.noarch 59/175 2026-02-22T23:02:29.836 INFO:teuthology.orchestra.run.trial168.stdout: Verifying : cephadm-2:19.2.3-689.gf90f4eae.el9.noarch 60/175 2026-02-22T23:02:29.836 INFO:teuthology.orchestra.run.trial168.stdout: Verifying : cryptsetup-2.8.1-3.el9.x86_64 61/175 2026-02-22T23:02:29.836 INFO:teuthology.orchestra.run.trial168.stdout: Verifying : ledmon-libs-1.1.0-3.el9.x86_64 62/175 2026-02-22T23:02:29.836 INFO:teuthology.orchestra.run.trial168.stdout: Verifying : libconfig-1.7.2-9.el9.x86_64 63/175 2026-02-22T23:02:29.836 INFO:teuthology.orchestra.run.trial168.stdout: Verifying : libgfortran-11.5.0-14.el9.x86_64 64/175 2026-02-22T23:02:29.836 INFO:teuthology.orchestra.run.trial168.stdout: Verifying : libquadmath-11.5.0-14.el9.x86_64 65/175 2026-02-22T23:02:29.836 INFO:teuthology.orchestra.run.trial168.stdout: Verifying : mailcap-2.1.49-5.el9.noarch 66/175 2026-02-22T23:02:29.836 INFO:teuthology.orchestra.run.trial168.stdout: Verifying : nvme-cli-2.16-1.el9.x86_64 67/175 2026-02-22T23:02:29.836 INFO:teuthology.orchestra.run.trial168.stdout: Verifying : pciutils-3.7.0-7.el9.x86_64 68/175 2026-02-22T23:02:29.836 INFO:teuthology.orchestra.run.trial168.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 69/175 2026-02-22T23:02:29.836 INFO:teuthology.orchestra.run.trial168.stdout: Verifying : python3-cryptography-36.0.1-5.el9.x86_64 70/175 2026-02-22T23:02:29.836 INFO:teuthology.orchestra.run.trial168.stdout: Verifying : python3-ply-3.11-14.el9.noarch 71/175 2026-02-22T23:02:29.836 INFO:teuthology.orchestra.run.trial168.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 72/175 2026-02-22T23:02:29.836 INFO:teuthology.orchestra.run.trial168.stdout: Verifying : python3-pyparsing-2.4.7-9.el9.noarch 73/175 2026-02-22T23:02:29.836 INFO:teuthology.orchestra.run.trial168.stdout: Verifying : python3-requests-2.25.1-10.el9.noarch 74/175 2026-02-22T23:02:29.836 INFO:teuthology.orchestra.run.trial168.stdout: Verifying : python3-urllib3-1.26.5-7.el9.noarch 75/175 2026-02-22T23:02:29.836 INFO:teuthology.orchestra.run.trial168.stdout: Verifying : unzip-6.0-59.el9.x86_64 76/175 2026-02-22T23:02:29.836 INFO:teuthology.orchestra.run.trial168.stdout: Verifying : zip-3.0-35.el9.x86_64 77/175 2026-02-22T23:02:29.836 INFO:teuthology.orchestra.run.trial168.stdout: Verifying : boost-program-options-1.75.0-13.el9.x86_64 78/175 2026-02-22T23:02:29.836 INFO:teuthology.orchestra.run.trial168.stdout: Verifying : flexiblas-3.0.4-9.el9.x86_64 79/175 2026-02-22T23:02:29.836 INFO:teuthology.orchestra.run.trial168.stdout: Verifying : flexiblas-netlib-3.0.4-9.el9.x86_64 80/175 2026-02-22T23:02:29.836 INFO:teuthology.orchestra.run.trial168.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-9.el9.x86_64 81/175 2026-02-22T23:02:29.836 INFO:teuthology.orchestra.run.trial168.stdout: Verifying : libnbd-1.20.3-4.el9.x86_64 82/175 2026-02-22T23:02:29.836 INFO:teuthology.orchestra.run.trial168.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 83/175 2026-02-22T23:02:29.836 INFO:teuthology.orchestra.run.trial168.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 84/175 2026-02-22T23:02:29.836 INFO:teuthology.orchestra.run.trial168.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 85/175 2026-02-22T23:02:29.836 INFO:teuthology.orchestra.run.trial168.stdout: Verifying : libstoragemgmt-1.10.1-1.el9.x86_64 86/175 2026-02-22T23:02:29.836 INFO:teuthology.orchestra.run.trial168.stdout: Verifying : libxslt-1.1.34-12.el9.x86_64 87/175 2026-02-22T23:02:29.837 INFO:teuthology.orchestra.run.trial168.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 88/175 2026-02-22T23:02:29.837 INFO:teuthology.orchestra.run.trial168.stdout: Verifying : lua-5.4.4-4.el9.x86_64 89/175 2026-02-22T23:02:29.837 INFO:teuthology.orchestra.run.trial168.stdout: Verifying : openblas-0.3.29-1.el9.x86_64 90/175 2026-02-22T23:02:29.837 INFO:teuthology.orchestra.run.trial168.stdout: Verifying : openblas-openmp-0.3.29-1.el9.x86_64 91/175 2026-02-22T23:02:29.837 INFO:teuthology.orchestra.run.trial168.stdout: Verifying : protobuf-3.14.0-17.el9.x86_64 92/175 2026-02-22T23:02:29.837 INFO:teuthology.orchestra.run.trial168.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 93/175 2026-02-22T23:02:29.837 INFO:teuthology.orchestra.run.trial168.stdout: Verifying : python3-devel-3.9.25-3.el9.x86_64 94/175 2026-02-22T23:02:29.837 INFO:teuthology.orchestra.run.trial168.stdout: Verifying : python3-iniconfig-1.1.1-7.el9.noarch 95/175 2026-02-22T23:02:29.837 INFO:teuthology.orchestra.run.trial168.stdout: Verifying : python3-jinja2-2.11.3-8.el9.noarch 96/175 2026-02-22T23:02:29.837 INFO:teuthology.orchestra.run.trial168.stdout: Verifying : python3-libstoragemgmt-1.10.1-1.el9.x86_64 97/175 2026-02-22T23:02:29.837 INFO:teuthology.orchestra.run.trial168.stdout: Verifying : python3-lxml-4.6.5-3.el9.x86_64 98/175 2026-02-22T23:02:29.837 INFO:teuthology.orchestra.run.trial168.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 99/175 2026-02-22T23:02:29.837 INFO:teuthology.orchestra.run.trial168.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 100/175 2026-02-22T23:02:29.837 INFO:teuthology.orchestra.run.trial168.stdout: Verifying : python3-numpy-1:1.23.5-2.el9.x86_64 101/175 2026-02-22T23:02:29.837 INFO:teuthology.orchestra.run.trial168.stdout: Verifying : python3-numpy-f2py-1:1.23.5-2.el9.x86_64 102/175 2026-02-22T23:02:29.837 INFO:teuthology.orchestra.run.trial168.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 103/175 2026-02-22T23:02:29.837 INFO:teuthology.orchestra.run.trial168.stdout: Verifying : python3-pluggy-0.13.1-7.el9.noarch 104/175 2026-02-22T23:02:29.837 INFO:teuthology.orchestra.run.trial168.stdout: Verifying : python3-protobuf-3.14.0-17.el9.noarch 105/175 2026-02-22T23:02:29.837 INFO:teuthology.orchestra.run.trial168.stdout: Verifying : python3-py-1.10.0-6.el9.noarch 106/175 2026-02-22T23:02:29.837 INFO:teuthology.orchestra.run.trial168.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 107/175 2026-02-22T23:02:29.837 INFO:teuthology.orchestra.run.trial168.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 108/175 2026-02-22T23:02:29.837 INFO:teuthology.orchestra.run.trial168.stdout: Verifying : python3-pytest-6.2.2-7.el9.noarch 109/175 2026-02-22T23:02:29.837 INFO:teuthology.orchestra.run.trial168.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 110/175 2026-02-22T23:02:29.837 INFO:teuthology.orchestra.run.trial168.stdout: Verifying : python3-scipy-1.9.3-2.el9.x86_64 111/175 2026-02-22T23:02:29.837 INFO:teuthology.orchestra.run.trial168.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 112/175 2026-02-22T23:02:29.837 INFO:teuthology.orchestra.run.trial168.stdout: Verifying : qatlib-25.08.0-2.el9.x86_64 113/175 2026-02-22T23:02:29.837 INFO:teuthology.orchestra.run.trial168.stdout: Verifying : qatlib-service-25.08.0-2.el9.x86_64 114/175 2026-02-22T23:02:29.837 INFO:teuthology.orchestra.run.trial168.stdout: Verifying : qatzip-libs-1.3.1-1.el9.x86_64 115/175 2026-02-22T23:02:29.837 INFO:teuthology.orchestra.run.trial168.stdout: Verifying : socat-1.7.4.1-8.el9.x86_64 116/175 2026-02-22T23:02:29.837 INFO:teuthology.orchestra.run.trial168.stdout: Verifying : xmlsec1-1.2.29-13.el9.x86_64 117/175 2026-02-22T23:02:29.837 INFO:teuthology.orchestra.run.trial168.stdout: Verifying : xmlsec1-openssl-1.2.29-13.el9.x86_64 118/175 2026-02-22T23:02:29.837 INFO:teuthology.orchestra.run.trial168.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 119/175 2026-02-22T23:02:29.837 INFO:teuthology.orchestra.run.trial168.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 120/175 2026-02-22T23:02:29.837 INFO:teuthology.orchestra.run.trial168.stdout: Verifying : protobuf-compiler-3.14.0-17.el9.x86_64 121/175 2026-02-22T23:02:29.837 INFO:teuthology.orchestra.run.trial168.stdout: Verifying : abseil-cpp-20211102.0-4.el9.x86_64 122/175 2026-02-22T23:02:29.837 INFO:teuthology.orchestra.run.trial168.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 123/175 2026-02-22T23:02:29.837 INFO:teuthology.orchestra.run.trial168.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 124/175 2026-02-22T23:02:29.837 INFO:teuthology.orchestra.run.trial168.stdout: Verifying : libarrow-9.0.0-15.el9.x86_64 125/175 2026-02-22T23:02:29.837 INFO:teuthology.orchestra.run.trial168.stdout: Verifying : libarrow-doc-9.0.0-15.el9.noarch 126/175 2026-02-22T23:02:29.837 INFO:teuthology.orchestra.run.trial168.stdout: Verifying : liboath-2.6.12-1.el9.x86_64 127/175 2026-02-22T23:02:29.838 INFO:teuthology.orchestra.run.trial168.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 128/175 2026-02-22T23:02:29.838 INFO:teuthology.orchestra.run.trial168.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 129/175 2026-02-22T23:02:29.838 INFO:teuthology.orchestra.run.trial168.stdout: Verifying : parquet-libs-9.0.0-15.el9.x86_64 130/175 2026-02-22T23:02:29.838 INFO:teuthology.orchestra.run.trial168.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 131/175 2026-02-22T23:02:29.838 INFO:teuthology.orchestra.run.trial168.stdout: Verifying : python3-autocommand-2.2.2-8.el9.noarch 132/175 2026-02-22T23:02:29.838 INFO:teuthology.orchestra.run.trial168.stdout: Verifying : python3-backports-tarfile-1.2.0-1.el9.noarch 133/175 2026-02-22T23:02:29.838 INFO:teuthology.orchestra.run.trial168.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 134/175 2026-02-22T23:02:29.838 INFO:teuthology.orchestra.run.trial168.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 135/175 2026-02-22T23:02:29.838 INFO:teuthology.orchestra.run.trial168.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 136/175 2026-02-22T23:02:29.838 INFO:teuthology.orchestra.run.trial168.stdout: Verifying : python3-cheroot-10.0.1-4.el9.noarch 137/175 2026-02-22T23:02:29.838 INFO:teuthology.orchestra.run.trial168.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 138/175 2026-02-22T23:02:29.838 INFO:teuthology.orchestra.run.trial168.stdout: Verifying : python3-google-auth-1:2.45.0-1.el9.noarch 139/175 2026-02-22T23:02:29.838 INFO:teuthology.orchestra.run.trial168.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 140/175 2026-02-22T23:02:29.838 INFO:teuthology.orchestra.run.trial168.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 141/175 2026-02-22T23:02:29.838 INFO:teuthology.orchestra.run.trial168.stdout: Verifying : python3-influxdb-5.3.1-1.el9.noarch 142/175 2026-02-22T23:02:29.838 INFO:teuthology.orchestra.run.trial168.stdout: Verifying : python3-isodate-0.6.1-3.el9.noarch 143/175 2026-02-22T23:02:29.838 INFO:teuthology.orchestra.run.trial168.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 144/175 2026-02-22T23:02:29.838 INFO:teuthology.orchestra.run.trial168.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 145/175 2026-02-22T23:02:29.838 INFO:teuthology.orchestra.run.trial168.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 146/175 2026-02-22T23:02:29.838 INFO:teuthology.orchestra.run.trial168.stdout: Verifying : python3-jaraco-context-6.0.1-3.el9.noarch 147/175 2026-02-22T23:02:29.838 INFO:teuthology.orchestra.run.trial168.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 148/175 2026-02-22T23:02:29.838 INFO:teuthology.orchestra.run.trial168.stdout: Verifying : python3-jaraco-text-4.0.0-2.el9.noarch 149/175 2026-02-22T23:02:29.838 INFO:teuthology.orchestra.run.trial168.stdout: Verifying : python3-kubernetes-1:26.1.0-3.el9.noarch 150/175 2026-02-22T23:02:29.838 INFO:teuthology.orchestra.run.trial168.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 151/175 2026-02-22T23:02:29.838 INFO:teuthology.orchestra.run.trial168.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 152/175 2026-02-22T23:02:29.838 INFO:teuthology.orchestra.run.trial168.stdout: Verifying : python3-msgpack-1.0.3-2.el9.x86_64 153/175 2026-02-22T23:02:29.838 INFO:teuthology.orchestra.run.trial168.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 154/175 2026-02-22T23:02:29.838 INFO:teuthology.orchestra.run.trial168.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 155/175 2026-02-22T23:02:29.838 INFO:teuthology.orchestra.run.trial168.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 156/175 2026-02-22T23:02:29.838 INFO:teuthology.orchestra.run.trial168.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 157/175 2026-02-22T23:02:29.838 INFO:teuthology.orchestra.run.trial168.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 158/175 2026-02-22T23:02:29.838 INFO:teuthology.orchestra.run.trial168.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 159/175 2026-02-22T23:02:29.838 INFO:teuthology.orchestra.run.trial168.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 160/175 2026-02-22T23:02:29.838 INFO:teuthology.orchestra.run.trial168.stdout: Verifying : python3-saml-1.16.0-1.el9.noarch 161/175 2026-02-22T23:02:29.838 INFO:teuthology.orchestra.run.trial168.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 162/175 2026-02-22T23:02:29.838 INFO:teuthology.orchestra.run.trial168.stdout: Verifying : python3-typing-extensions-4.15.0-1.el9.noarch 163/175 2026-02-22T23:02:29.838 INFO:teuthology.orchestra.run.trial168.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 164/175 2026-02-22T23:02:29.838 INFO:teuthology.orchestra.run.trial168.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 165/175 2026-02-22T23:02:29.838 INFO:teuthology.orchestra.run.trial168.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 166/175 2026-02-22T23:02:29.838 INFO:teuthology.orchestra.run.trial168.stdout: Verifying : python3-xmlsec-1.3.13-1.el9.x86_64 167/175 2026-02-22T23:02:29.839 INFO:teuthology.orchestra.run.trial168.stdout: Verifying : python3-xmltodict-0.12.0-15.el9.noarch 168/175 2026-02-22T23:02:29.839 INFO:teuthology.orchestra.run.trial168.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 169/175 2026-02-22T23:02:29.839 INFO:teuthology.orchestra.run.trial168.stdout: Verifying : re2-1:20211101-20.el9.x86_64 170/175 2026-02-22T23:02:29.839 INFO:teuthology.orchestra.run.trial168.stdout: Verifying : thrift-0.15.0-4.el9.x86_64 171/175 2026-02-22T23:02:29.839 INFO:teuthology.orchestra.run.trial168.stdout: Verifying : librados2-2:19.2.3-689.gf90f4eae.el9.x86_64 172/175 2026-02-22T23:02:29.839 INFO:teuthology.orchestra.run.trial168.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 173/175 2026-02-22T23:02:29.839 INFO:teuthology.orchestra.run.trial168.stdout: Verifying : librbd1-2:19.2.3-689.gf90f4eae.el9.x86_64 174/175 2026-02-22T23:02:29.913 INFO:teuthology.orchestra.run.trial168.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 175/175 2026-02-22T23:02:29.913 INFO:teuthology.orchestra.run.trial168.stdout: 2026-02-22T23:02:29.913 INFO:teuthology.orchestra.run.trial168.stdout:Upgraded: 2026-02-22T23:02:29.914 INFO:teuthology.orchestra.run.trial168.stdout: librados2-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-22T23:02:29.914 INFO:teuthology.orchestra.run.trial168.stdout: librbd1-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-22T23:02:29.914 INFO:teuthology.orchestra.run.trial168.stdout:Installed: 2026-02-22T23:02:29.914 INFO:teuthology.orchestra.run.trial168.stdout: abseil-cpp-20211102.0-4.el9.x86_64 2026-02-22T23:02:29.914 INFO:teuthology.orchestra.run.trial168.stdout: boost-program-options-1.75.0-13.el9.x86_64 2026-02-22T23:02:29.914 INFO:teuthology.orchestra.run.trial168.stdout: ceph-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-22T23:02:29.914 INFO:teuthology.orchestra.run.trial168.stdout: ceph-base-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-22T23:02:29.914 INFO:teuthology.orchestra.run.trial168.stdout: ceph-base-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-22T23:02:29.914 INFO:teuthology.orchestra.run.trial168.stdout: ceph-common-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-22T23:02:29.914 INFO:teuthology.orchestra.run.trial168.stdout: ceph-common-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-22T23:02:29.914 INFO:teuthology.orchestra.run.trial168.stdout: ceph-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-22T23:02:29.914 INFO:teuthology.orchestra.run.trial168.stdout: ceph-debugsource-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-22T23:02:29.915 INFO:teuthology.orchestra.run.trial168.stdout: ceph-fuse-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-22T23:02:29.915 INFO:teuthology.orchestra.run.trial168.stdout: ceph-fuse-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-22T23:02:29.915 INFO:teuthology.orchestra.run.trial168.stdout: ceph-grafana-dashboards-2:19.2.3-689.gf90f4eae.el9.noarch 2026-02-22T23:02:29.915 INFO:teuthology.orchestra.run.trial168.stdout: ceph-immutable-object-cache-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-22T23:02:29.915 INFO:teuthology.orchestra.run.trial168.stdout: ceph-immutable-object-cache-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-22T23:02:29.915 INFO:teuthology.orchestra.run.trial168.stdout: ceph-mds-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-22T23:02:29.915 INFO:teuthology.orchestra.run.trial168.stdout: ceph-mds-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-22T23:02:29.915 INFO:teuthology.orchestra.run.trial168.stdout: ceph-mgr-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-22T23:02:29.915 INFO:teuthology.orchestra.run.trial168.stdout: ceph-mgr-cephadm-2:19.2.3-689.gf90f4eae.el9.noarch 2026-02-22T23:02:29.915 INFO:teuthology.orchestra.run.trial168.stdout: ceph-mgr-dashboard-2:19.2.3-689.gf90f4eae.el9.noarch 2026-02-22T23:02:29.915 INFO:teuthology.orchestra.run.trial168.stdout: ceph-mgr-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-22T23:02:29.916 INFO:teuthology.orchestra.run.trial168.stdout: ceph-mgr-diskprediction-local-2:19.2.3-689.gf90f4eae.el9.noarch 2026-02-22T23:02:29.916 INFO:teuthology.orchestra.run.trial168.stdout: ceph-mgr-k8sevents-2:19.2.3-689.gf90f4eae.el9.noarch 2026-02-22T23:02:29.916 INFO:teuthology.orchestra.run.trial168.stdout: ceph-mgr-modules-core-2:19.2.3-689.gf90f4eae.el9.noarch 2026-02-22T23:02:29.916 INFO:teuthology.orchestra.run.trial168.stdout: ceph-mgr-rook-2:19.2.3-689.gf90f4eae.el9.noarch 2026-02-22T23:02:29.916 INFO:teuthology.orchestra.run.trial168.stdout: ceph-mon-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-22T23:02:29.916 INFO:teuthology.orchestra.run.trial168.stdout: ceph-mon-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-22T23:02:29.916 INFO:teuthology.orchestra.run.trial168.stdout: ceph-osd-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-22T23:02:29.916 INFO:teuthology.orchestra.run.trial168.stdout: ceph-osd-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-22T23:02:29.916 INFO:teuthology.orchestra.run.trial168.stdout: ceph-prometheus-alerts-2:19.2.3-689.gf90f4eae.el9.noarch 2026-02-22T23:02:29.916 INFO:teuthology.orchestra.run.trial168.stdout: ceph-radosgw-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-22T23:02:29.916 INFO:teuthology.orchestra.run.trial168.stdout: ceph-radosgw-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-22T23:02:29.916 INFO:teuthology.orchestra.run.trial168.stdout: ceph-selinux-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-22T23:02:29.917 INFO:teuthology.orchestra.run.trial168.stdout: ceph-test-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-22T23:02:29.917 INFO:teuthology.orchestra.run.trial168.stdout: ceph-test-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-22T23:02:29.917 INFO:teuthology.orchestra.run.trial168.stdout: ceph-volume-2:19.2.3-689.gf90f4eae.el9.noarch 2026-02-22T23:02:29.917 INFO:teuthology.orchestra.run.trial168.stdout: cephadm-2:19.2.3-689.gf90f4eae.el9.noarch 2026-02-22T23:02:29.917 INFO:teuthology.orchestra.run.trial168.stdout: cryptsetup-2.8.1-3.el9.x86_64 2026-02-22T23:02:29.917 INFO:teuthology.orchestra.run.trial168.stdout: flexiblas-3.0.4-9.el9.x86_64 2026-02-22T23:02:29.917 INFO:teuthology.orchestra.run.trial168.stdout: flexiblas-netlib-3.0.4-9.el9.x86_64 2026-02-22T23:02:29.917 INFO:teuthology.orchestra.run.trial168.stdout: flexiblas-openblas-openmp-3.0.4-9.el9.x86_64 2026-02-22T23:02:29.917 INFO:teuthology.orchestra.run.trial168.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2026-02-22T23:02:29.917 INFO:teuthology.orchestra.run.trial168.stdout: grpc-data-1.46.7-10.el9.noarch 2026-02-22T23:02:29.917 INFO:teuthology.orchestra.run.trial168.stdout: ledmon-libs-1.1.0-3.el9.x86_64 2026-02-22T23:02:29.917 INFO:teuthology.orchestra.run.trial168.stdout: libarrow-9.0.0-15.el9.x86_64 2026-02-22T23:02:29.918 INFO:teuthology.orchestra.run.trial168.stdout: libarrow-doc-9.0.0-15.el9.noarch 2026-02-22T23:02:29.918 INFO:teuthology.orchestra.run.trial168.stdout: libcephfs-devel-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-22T23:02:29.918 INFO:teuthology.orchestra.run.trial168.stdout: libcephfs2-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-22T23:02:29.918 INFO:teuthology.orchestra.run.trial168.stdout: libcephfs2-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-22T23:02:29.918 INFO:teuthology.orchestra.run.trial168.stdout: libcephsqlite-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-22T23:02:29.918 INFO:teuthology.orchestra.run.trial168.stdout: libconfig-1.7.2-9.el9.x86_64 2026-02-22T23:02:29.918 INFO:teuthology.orchestra.run.trial168.stdout: libgfortran-11.5.0-14.el9.x86_64 2026-02-22T23:02:29.918 INFO:teuthology.orchestra.run.trial168.stdout: libnbd-1.20.3-4.el9.x86_64 2026-02-22T23:02:29.918 INFO:teuthology.orchestra.run.trial168.stdout: liboath-2.6.12-1.el9.x86_64 2026-02-22T23:02:29.918 INFO:teuthology.orchestra.run.trial168.stdout: libpmemobj-1.12.1-1.el9.x86_64 2026-02-22T23:02:29.918 INFO:teuthology.orchestra.run.trial168.stdout: libquadmath-11.5.0-14.el9.x86_64 2026-02-22T23:02:29.918 INFO:teuthology.orchestra.run.trial168.stdout: librabbitmq-0.11.0-7.el9.x86_64 2026-02-22T23:02:29.918 INFO:teuthology.orchestra.run.trial168.stdout: librados-devel-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-22T23:02:29.919 INFO:teuthology.orchestra.run.trial168.stdout: librados-devel-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-22T23:02:29.919 INFO:teuthology.orchestra.run.trial168.stdout: librados2-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-22T23:02:29.919 INFO:teuthology.orchestra.run.trial168.stdout: libradosstriper1-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-22T23:02:29.919 INFO:teuthology.orchestra.run.trial168.stdout: librbd1-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-22T23:02:29.919 INFO:teuthology.orchestra.run.trial168.stdout: librdkafka-1.6.1-102.el9.x86_64 2026-02-22T23:02:29.919 INFO:teuthology.orchestra.run.trial168.stdout: librgw2-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-22T23:02:29.919 INFO:teuthology.orchestra.run.trial168.stdout: libstoragemgmt-1.10.1-1.el9.x86_64 2026-02-22T23:02:29.919 INFO:teuthology.orchestra.run.trial168.stdout: libunwind-1.6.2-1.el9.x86_64 2026-02-22T23:02:29.919 INFO:teuthology.orchestra.run.trial168.stdout: libxslt-1.1.34-12.el9.x86_64 2026-02-22T23:02:29.919 INFO:teuthology.orchestra.run.trial168.stdout: lttng-ust-2.12.0-6.el9.x86_64 2026-02-22T23:02:29.919 INFO:teuthology.orchestra.run.trial168.stdout: lua-5.4.4-4.el9.x86_64 2026-02-22T23:02:29.919 INFO:teuthology.orchestra.run.trial168.stdout: lua-devel-5.4.4-4.el9.x86_64 2026-02-22T23:02:29.920 INFO:teuthology.orchestra.run.trial168.stdout: luarocks-3.9.2-5.el9.noarch 2026-02-22T23:02:29.920 INFO:teuthology.orchestra.run.trial168.stdout: mailcap-2.1.49-5.el9.noarch 2026-02-22T23:02:29.920 INFO:teuthology.orchestra.run.trial168.stdout: nvme-cli-2.16-1.el9.x86_64 2026-02-22T23:02:29.920 INFO:teuthology.orchestra.run.trial168.stdout: openblas-0.3.29-1.el9.x86_64 2026-02-22T23:02:29.920 INFO:teuthology.orchestra.run.trial168.stdout: openblas-openmp-0.3.29-1.el9.x86_64 2026-02-22T23:02:29.920 INFO:teuthology.orchestra.run.trial168.stdout: parquet-libs-9.0.0-15.el9.x86_64 2026-02-22T23:02:29.920 INFO:teuthology.orchestra.run.trial168.stdout: pciutils-3.7.0-7.el9.x86_64 2026-02-22T23:02:29.920 INFO:teuthology.orchestra.run.trial168.stdout: protobuf-3.14.0-17.el9.x86_64 2026-02-22T23:02:29.920 INFO:teuthology.orchestra.run.trial168.stdout: protobuf-compiler-3.14.0-17.el9.x86_64 2026-02-22T23:02:29.920 INFO:teuthology.orchestra.run.trial168.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2026-02-22T23:02:29.920 INFO:teuthology.orchestra.run.trial168.stdout: python3-autocommand-2.2.2-8.el9.noarch 2026-02-22T23:02:29.920 INFO:teuthology.orchestra.run.trial168.stdout: python3-babel-2.9.1-2.el9.noarch 2026-02-22T23:02:29.920 INFO:teuthology.orchestra.run.trial168.stdout: python3-backports-tarfile-1.2.0-1.el9.noarch 2026-02-22T23:02:29.921 INFO:teuthology.orchestra.run.trial168.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2026-02-22T23:02:29.921 INFO:teuthology.orchestra.run.trial168.stdout: python3-cachetools-4.2.4-1.el9.noarch 2026-02-22T23:02:29.921 INFO:teuthology.orchestra.run.trial168.stdout: python3-ceph-argparse-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-22T23:02:29.921 INFO:teuthology.orchestra.run.trial168.stdout: python3-ceph-common-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-22T23:02:29.921 INFO:teuthology.orchestra.run.trial168.stdout: python3-cephfs-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-22T23:02:29.921 INFO:teuthology.orchestra.run.trial168.stdout: python3-cephfs-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-22T23:02:29.921 INFO:teuthology.orchestra.run.trial168.stdout: python3-certifi-2023.05.07-4.el9.noarch 2026-02-22T23:02:29.921 INFO:teuthology.orchestra.run.trial168.stdout: python3-cffi-1.14.5-5.el9.x86_64 2026-02-22T23:02:29.921 INFO:teuthology.orchestra.run.trial168.stdout: python3-cheroot-10.0.1-4.el9.noarch 2026-02-22T23:02:29.921 INFO:teuthology.orchestra.run.trial168.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2026-02-22T23:02:29.921 INFO:teuthology.orchestra.run.trial168.stdout: python3-cryptography-36.0.1-5.el9.x86_64 2026-02-22T23:02:29.921 INFO:teuthology.orchestra.run.trial168.stdout: python3-devel-3.9.25-3.el9.x86_64 2026-02-22T23:02:29.922 INFO:teuthology.orchestra.run.trial168.stdout: python3-google-auth-1:2.45.0-1.el9.noarch 2026-02-22T23:02:29.922 INFO:teuthology.orchestra.run.trial168.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2026-02-22T23:02:29.922 INFO:teuthology.orchestra.run.trial168.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2026-02-22T23:02:29.922 INFO:teuthology.orchestra.run.trial168.stdout: python3-influxdb-5.3.1-1.el9.noarch 2026-02-22T23:02:29.922 INFO:teuthology.orchestra.run.trial168.stdout: python3-iniconfig-1.1.1-7.el9.noarch 2026-02-22T23:02:29.922 INFO:teuthology.orchestra.run.trial168.stdout: python3-isodate-0.6.1-3.el9.noarch 2026-02-22T23:02:29.922 INFO:teuthology.orchestra.run.trial168.stdout: python3-jaraco-8.2.1-3.el9.noarch 2026-02-22T23:02:29.922 INFO:teuthology.orchestra.run.trial168.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2026-02-22T23:02:29.922 INFO:teuthology.orchestra.run.trial168.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2026-02-22T23:02:29.922 INFO:teuthology.orchestra.run.trial168.stdout: python3-jaraco-context-6.0.1-3.el9.noarch 2026-02-22T23:02:29.922 INFO:teuthology.orchestra.run.trial168.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2026-02-22T23:02:29.922 INFO:teuthology.orchestra.run.trial168.stdout: python3-jaraco-text-4.0.0-2.el9.noarch 2026-02-22T23:02:29.923 INFO:teuthology.orchestra.run.trial168.stdout: python3-jinja2-2.11.3-8.el9.noarch 2026-02-22T23:02:29.923 INFO:teuthology.orchestra.run.trial168.stdout: python3-kubernetes-1:26.1.0-3.el9.noarch 2026-02-22T23:02:29.923 INFO:teuthology.orchestra.run.trial168.stdout: python3-libstoragemgmt-1.10.1-1.el9.x86_64 2026-02-22T23:02:29.923 INFO:teuthology.orchestra.run.trial168.stdout: python3-logutils-0.3.5-21.el9.noarch 2026-02-22T23:02:29.923 INFO:teuthology.orchestra.run.trial168.stdout: python3-lxml-4.6.5-3.el9.x86_64 2026-02-22T23:02:29.923 INFO:teuthology.orchestra.run.trial168.stdout: python3-mako-1.1.4-6.el9.noarch 2026-02-22T23:02:29.923 INFO:teuthology.orchestra.run.trial168.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2026-02-22T23:02:29.923 INFO:teuthology.orchestra.run.trial168.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2026-02-22T23:02:29.923 INFO:teuthology.orchestra.run.trial168.stdout: python3-msgpack-1.0.3-2.el9.x86_64 2026-02-22T23:02:29.923 INFO:teuthology.orchestra.run.trial168.stdout: python3-natsort-7.1.1-5.el9.noarch 2026-02-22T23:02:29.923 INFO:teuthology.orchestra.run.trial168.stdout: python3-numpy-1:1.23.5-2.el9.x86_64 2026-02-22T23:02:29.923 INFO:teuthology.orchestra.run.trial168.stdout: python3-numpy-f2py-1:1.23.5-2.el9.x86_64 2026-02-22T23:02:29.923 INFO:teuthology.orchestra.run.trial168.stdout: python3-packaging-20.9-5.el9.noarch 2026-02-22T23:02:29.924 INFO:teuthology.orchestra.run.trial168.stdout: python3-pecan-1.4.2-3.el9.noarch 2026-02-22T23:02:29.924 INFO:teuthology.orchestra.run.trial168.stdout: python3-pluggy-0.13.1-7.el9.noarch 2026-02-22T23:02:29.924 INFO:teuthology.orchestra.run.trial168.stdout: python3-ply-3.11-14.el9.noarch 2026-02-22T23:02:29.924 INFO:teuthology.orchestra.run.trial168.stdout: python3-portend-3.1.0-2.el9.noarch 2026-02-22T23:02:29.924 INFO:teuthology.orchestra.run.trial168.stdout: python3-protobuf-3.14.0-17.el9.noarch 2026-02-22T23:02:29.924 INFO:teuthology.orchestra.run.trial168.stdout: python3-py-1.10.0-6.el9.noarch 2026-02-22T23:02:29.924 INFO:teuthology.orchestra.run.trial168.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2026-02-22T23:02:29.924 INFO:teuthology.orchestra.run.trial168.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2026-02-22T23:02:29.924 INFO:teuthology.orchestra.run.trial168.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2026-02-22T23:02:29.924 INFO:teuthology.orchestra.run.trial168.stdout: python3-pycparser-2.20-6.el9.noarch 2026-02-22T23:02:29.924 INFO:teuthology.orchestra.run.trial168.stdout: python3-pyparsing-2.4.7-9.el9.noarch 2026-02-22T23:02:29.924 INFO:teuthology.orchestra.run.trial168.stdout: python3-pytest-6.2.2-7.el9.noarch 2026-02-22T23:02:29.924 INFO:teuthology.orchestra.run.trial168.stdout: python3-rados-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-22T23:02:29.925 INFO:teuthology.orchestra.run.trial168.stdout: python3-rados-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-22T23:02:29.925 INFO:teuthology.orchestra.run.trial168.stdout: python3-rbd-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-22T23:02:29.925 INFO:teuthology.orchestra.run.trial168.stdout: python3-rbd-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-22T23:02:29.925 INFO:teuthology.orchestra.run.trial168.stdout: python3-repoze-lru-0.7-16.el9.noarch 2026-02-22T23:02:29.925 INFO:teuthology.orchestra.run.trial168.stdout: python3-requests-2.25.1-10.el9.noarch 2026-02-22T23:02:29.925 INFO:teuthology.orchestra.run.trial168.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2026-02-22T23:02:29.925 INFO:teuthology.orchestra.run.trial168.stdout: python3-rgw-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-22T23:02:29.925 INFO:teuthology.orchestra.run.trial168.stdout: python3-rgw-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-22T23:02:29.925 INFO:teuthology.orchestra.run.trial168.stdout: python3-routes-2.5.1-5.el9.noarch 2026-02-22T23:02:29.925 INFO:teuthology.orchestra.run.trial168.stdout: python3-rsa-4.9-2.el9.noarch 2026-02-22T23:02:29.925 INFO:teuthology.orchestra.run.trial168.stdout: python3-saml-1.16.0-1.el9.noarch 2026-02-22T23:02:29.925 INFO:teuthology.orchestra.run.trial168.stdout: python3-scipy-1.9.3-2.el9.x86_64 2026-02-22T23:02:29.925 INFO:teuthology.orchestra.run.trial168.stdout: python3-tempora-5.0.0-2.el9.noarch 2026-02-22T23:02:29.926 INFO:teuthology.orchestra.run.trial168.stdout: python3-toml-0.10.2-6.el9.noarch 2026-02-22T23:02:29.926 INFO:teuthology.orchestra.run.trial168.stdout: python3-typing-extensions-4.15.0-1.el9.noarch 2026-02-22T23:02:29.926 INFO:teuthology.orchestra.run.trial168.stdout: python3-urllib3-1.26.5-7.el9.noarch 2026-02-22T23:02:29.926 INFO:teuthology.orchestra.run.trial168.stdout: python3-webob-1.8.8-2.el9.noarch 2026-02-22T23:02:29.926 INFO:teuthology.orchestra.run.trial168.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2026-02-22T23:02:29.926 INFO:teuthology.orchestra.run.trial168.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2026-02-22T23:02:29.926 INFO:teuthology.orchestra.run.trial168.stdout: python3-xmlsec-1.3.13-1.el9.x86_64 2026-02-22T23:02:29.926 INFO:teuthology.orchestra.run.trial168.stdout: python3-xmltodict-0.12.0-15.el9.noarch 2026-02-22T23:02:29.926 INFO:teuthology.orchestra.run.trial168.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2026-02-22T23:02:29.926 INFO:teuthology.orchestra.run.trial168.stdout: qatlib-25.08.0-2.el9.x86_64 2026-02-22T23:02:29.926 INFO:teuthology.orchestra.run.trial168.stdout: qatlib-service-25.08.0-2.el9.x86_64 2026-02-22T23:02:29.926 INFO:teuthology.orchestra.run.trial168.stdout: qatzip-libs-1.3.1-1.el9.x86_64 2026-02-22T23:02:29.927 INFO:teuthology.orchestra.run.trial168.stdout: rbd-fuse-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-22T23:02:29.927 INFO:teuthology.orchestra.run.trial168.stdout: rbd-fuse-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-22T23:02:29.927 INFO:teuthology.orchestra.run.trial168.stdout: rbd-mirror-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-22T23:02:29.927 INFO:teuthology.orchestra.run.trial168.stdout: rbd-mirror-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-22T23:02:29.927 INFO:teuthology.orchestra.run.trial168.stdout: rbd-nbd-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-22T23:02:29.927 INFO:teuthology.orchestra.run.trial168.stdout: rbd-nbd-debuginfo-2:19.2.3-689.gf90f4eae.el9.x86_64 2026-02-22T23:02:29.927 INFO:teuthology.orchestra.run.trial168.stdout: re2-1:20211101-20.el9.x86_64 2026-02-22T23:02:29.927 INFO:teuthology.orchestra.run.trial168.stdout: socat-1.7.4.1-8.el9.x86_64 2026-02-22T23:02:29.927 INFO:teuthology.orchestra.run.trial168.stdout: thrift-0.15.0-4.el9.x86_64 2026-02-22T23:02:29.927 INFO:teuthology.orchestra.run.trial168.stdout: unzip-6.0-59.el9.x86_64 2026-02-22T23:02:29.927 INFO:teuthology.orchestra.run.trial168.stdout: xmlsec1-1.2.29-13.el9.x86_64 2026-02-22T23:02:29.927 INFO:teuthology.orchestra.run.trial168.stdout: xmlsec1-openssl-1.2.29-13.el9.x86_64 2026-02-22T23:02:29.927 INFO:teuthology.orchestra.run.trial168.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2026-02-22T23:02:29.928 INFO:teuthology.orchestra.run.trial168.stdout: zip-3.0-35.el9.x86_64 2026-02-22T23:02:29.928 INFO:teuthology.orchestra.run.trial168.stdout: 2026-02-22T23:02:29.928 INFO:teuthology.orchestra.run.trial168.stdout:Complete! 2026-02-22T23:02:30.226 DEBUG:teuthology.parallel:result is None 2026-02-22T23:02:30.227 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=f90f4eaebb28b619cf76e80cbbe997656da69579 2026-02-22T23:02:30.400 DEBUG:teuthology.orchestra.run.trial168:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2026-02-22T23:02:30.422 INFO:teuthology.orchestra.run.trial168.stdout:19.2.3-689.gf90f4eae.el9 2026-02-22T23:02:30.423 INFO:teuthology.packaging:The installed version of ceph is 19.2.3-689.gf90f4eae.el9 2026-02-22T23:02:30.423 INFO:teuthology.task.install:The correct ceph version 19.2.3-689.gf90f4eae is installed. 2026-02-22T23:02:30.425 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2026-02-22T23:02:30.425 DEBUG:teuthology.orchestra.run.trial168:> set -ex 2026-02-22T23:02:30.425 DEBUG:teuthology.orchestra.run.trial168:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2026-02-22T23:02:30.488 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2026-02-22T23:02:30.488 DEBUG:teuthology.orchestra.run.trial168:> set -ex 2026-02-22T23:02:30.488 DEBUG:teuthology.orchestra.run.trial168:> sudo dd of=/usr/bin/daemon-helper 2026-02-22T23:02:30.553 DEBUG:teuthology.orchestra.run.trial168:> sudo chmod a=rx -- /usr/bin/daemon-helper 2026-02-22T23:02:30.618 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2026-02-22T23:02:30.618 DEBUG:teuthology.orchestra.run.trial168:> set -ex 2026-02-22T23:02:30.618 DEBUG:teuthology.orchestra.run.trial168:> sudo dd of=/usr/bin/adjust-ulimits 2026-02-22T23:02:30.682 DEBUG:teuthology.orchestra.run.trial168:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2026-02-22T23:02:30.747 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2026-02-22T23:02:30.748 DEBUG:teuthology.orchestra.run.trial168:> set -ex 2026-02-22T23:02:30.748 DEBUG:teuthology.orchestra.run.trial168:> sudo dd of=/usr/bin/stdin-killer 2026-02-22T23:02:30.813 DEBUG:teuthology.orchestra.run.trial168:> sudo chmod a=rx -- /usr/bin/stdin-killer 2026-02-22T23:02:30.877 INFO:teuthology.run_tasks:Running task ceph... 2026-02-22T23:02:30.995 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2026-02-22T23:02:30.995 DEBUG:teuthology.orchestra.run.trial168:> sudo chmod 777 /var/log/ceph 2026-02-22T23:02:31.019 INFO:tasks.ceph:Disabling ceph logrotate... 2026-02-22T23:02:31.020 DEBUG:teuthology.orchestra.run.trial168:> sudo rm -f -- /etc/logrotate.d/ceph 2026-02-22T23:02:31.084 INFO:tasks.ceph:Creating extra log directories... 2026-02-22T23:02:31.084 DEBUG:teuthology.orchestra.run.trial168:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2026-02-22T23:02:31.159 INFO:tasks.ceph:Creating ceph cluster ceph... 2026-02-22T23:02:31.160 INFO:tasks.ceph:config {'conf': {'global': {'mon warn on pool no app': False}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'bluefs allocator': 'hybrid', 'bluestore allocator': 'hybrid', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd shutdown pgref assert': True}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2026-02-22T23:02:31.160 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/mnt/teuthology/yuriw-2026-02-22_19:35:38-rbd-wip-yuri6-testing-2026-02-22-1629-squid-distro-default-trial/64300', 'branch': 'wip-yuri6-testing-2026-02-22-1629-squid', 'description': 'rbd/valgrind/{base/install centos_latest clusters/{fixed-1 openstack} conf/{disable-pool-app} objectstore/bluestore-hybrid validator/memcheck workloads/python_api_tests}', 'email': 'yweinste@redhat.com', 'first_in_suite': False, 'flavor': 'default', 'job_id': '64300', 'kernel': {'branch': 'distro', 'kdb': True, 'sha1': 'distro'}, 'last_in_suite': False, 'machine_type': 'trial', 'name': 'yuriw-2026-02-22_19:35:38-rbd-wip-yuri6-testing-2026-02-22-1629-squid-distro-default-trial', 'no_nested_subset': False, 'openstack': [{'volumes': {'count': 3, 'size': 30}}], 'os_type': 'centos', 'os_version': '9.stream', 'overrides': {'admin_socket': {'branch': 'wip-yuri6-testing-2026-02-22-1629-squid'}, 'ceph': {'conf': {'global': {'mon warn on pool no app': False}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'bluefs allocator': 'hybrid', 'bluestore allocator': 'hybrid', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd shutdown pgref assert': True}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)'], 'sha1': 'f90f4eaebb28b619cf76e80cbbe997656da69579'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'global': {'osd crush chooseleaf type': 0, 'osd pool default pg num': 128, 'osd pool default pgp num': 128, 'osd pool default size': 2}, 'mon': {}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'debuginfo': True, 'flavor': 'default', 'sha1': 'f90f4eaebb28b619cf76e80cbbe997656da69579'}, 'extra_system_packages': ['python3-pytest']}, 'rbd_fsx': {'valgrind': ['--tool=memcheck']}, 'selinux': {'allowlist': ['scontext=system_u:system_r:getty_t:s0']}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-yuri6-testing-2026-02-22-1629-squid', 'env': {'VALGRIND': '--tool=memcheck --leak-check=full'}, 'sha1': 'f90f4eaebb28b619cf76e80cbbe997656da69579'}}, 'owner': 'scheduled_yuriw@soko04.front.sepia.ceph.com', 'priority': 69, 'repo': 'https://github.com/ceph/ceph-ci.git', 'roles': [['mon.a', 'mgr.x', 'osd.0', 'osd.1', 'osd.2', 'client.0']], 'seed': 9196, 'sha1': 'f90f4eaebb28b619cf76e80cbbe997656da69579', 'sleep_before_teardown': 0, 'subset': '111/120000', 'suite': 'rbd', 'suite_branch': 'wip-yuri6-testing-2026-02-22-1629-squid', 'suite_path': '/home/teuthworker/src/github.com_ceph_ceph-c_f90f4eaebb28b619cf76e80cbbe997656da69579/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://github.com/ceph/ceph-ci.git', 'suite_sha1': 'f90f4eaebb28b619cf76e80cbbe997656da69579', 'targets': {'trial168.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHNIO79a5opEL90QIdZsj0tbsZXOM31K1RPGlxWiNLr18RNdFHu6GFQWSFmgCQIkcUzEd0LsquwlIvlBwJNKbHU='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'branch': 'distro', 'kdb': True, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': None}, {'ceph': None}, {'workunit': {'clients': {'client.0': ['rbd/test_librbd_python.sh']}, 'env': {'RBD_FEATURES': '1'}}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': []}, 'teuthology_branch': 'main', 'teuthology_sha1': 'fa17720d0088c3ac28e473468bfc79eeaff5cd38', 'timestamp': '2026-02-22_19:35:38', 'tube': 'trial', 'user': 'yuriw', 'verbose': True, 'worker_log': '/home/teuthworker/mnt/teuthology/worker_logs/dispatcher.trial.3318653'} 2026-02-22T23:02:31.161 DEBUG:teuthology.orchestra.run.trial168:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2026-02-22T23:02:31.206 DEBUG:teuthology.orchestra.run.trial168:> sudo install -d -m0777 -- /var/run/ceph 2026-02-22T23:02:31.270 DEBUG:teuthology.orchestra.run.trial168:> set -ex 2026-02-22T23:02:31.271 DEBUG:teuthology.orchestra.run.trial168:> dd if=/scratch_devs of=/dev/stdout 2026-02-22T23:02:31.330 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2026-02-22T23:02:31.331 DEBUG:teuthology.orchestra.run.trial168:> stat /dev/vg_nvme/lv_1 2026-02-22T23:02:31.388 INFO:teuthology.orchestra.run.trial168.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2026-02-22T23:02:31.388 INFO:teuthology.orchestra.run.trial168.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2026-02-22T23:02:31.388 INFO:teuthology.orchestra.run.trial168.stdout:Device: 6h/6d Inode: 1317 Links: 1 2026-02-22T23:02:31.388 INFO:teuthology.orchestra.run.trial168.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2026-02-22T23:02:31.389 INFO:teuthology.orchestra.run.trial168.stdout:Context: system_u:object_r:device_t:s0 2026-02-22T23:02:31.389 INFO:teuthology.orchestra.run.trial168.stdout:Access: 2026-02-22 23:02:26.611762933 +0000 2026-02-22T23:02:31.389 INFO:teuthology.orchestra.run.trial168.stdout:Modify: 2026-02-22 23:00:32.736411335 +0000 2026-02-22T23:02:31.389 INFO:teuthology.orchestra.run.trial168.stdout:Change: 2026-02-22 23:00:32.736411335 +0000 2026-02-22T23:02:31.389 INFO:teuthology.orchestra.run.trial168.stdout: Birth: 2026-02-22 23:00:32.736411335 +0000 2026-02-22T23:02:31.389 DEBUG:teuthology.orchestra.run.trial168:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2026-02-22T23:02:31.453 INFO:teuthology.orchestra.run.trial168.stderr:1+0 records in 2026-02-22T23:02:31.453 INFO:teuthology.orchestra.run.trial168.stderr:1+0 records out 2026-02-22T23:02:31.454 INFO:teuthology.orchestra.run.trial168.stderr:512 bytes copied, 0.000162402 s, 3.2 MB/s 2026-02-22T23:02:31.455 DEBUG:teuthology.orchestra.run.trial168:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2026-02-22T23:02:31.513 DEBUG:teuthology.orchestra.run.trial168:> stat /dev/vg_nvme/lv_2 2026-02-22T23:02:31.569 INFO:teuthology.orchestra.run.trial168.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2026-02-22T23:02:31.569 INFO:teuthology.orchestra.run.trial168.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2026-02-22T23:02:31.569 INFO:teuthology.orchestra.run.trial168.stdout:Device: 6h/6d Inode: 1306 Links: 1 2026-02-22T23:02:31.569 INFO:teuthology.orchestra.run.trial168.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2026-02-22T23:02:31.569 INFO:teuthology.orchestra.run.trial168.stdout:Context: system_u:object_r:device_t:s0 2026-02-22T23:02:31.569 INFO:teuthology.orchestra.run.trial168.stdout:Access: 2026-02-22 23:02:26.611762933 +0000 2026-02-22T23:02:31.569 INFO:teuthology.orchestra.run.trial168.stdout:Modify: 2026-02-22 23:00:32.735411326 +0000 2026-02-22T23:02:31.569 INFO:teuthology.orchestra.run.trial168.stdout:Change: 2026-02-22 23:00:32.735411326 +0000 2026-02-22T23:02:31.569 INFO:teuthology.orchestra.run.trial168.stdout: Birth: 2026-02-22 23:00:32.735411326 +0000 2026-02-22T23:02:31.569 DEBUG:teuthology.orchestra.run.trial168:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2026-02-22T23:02:31.631 INFO:teuthology.orchestra.run.trial168.stderr:1+0 records in 2026-02-22T23:02:31.632 INFO:teuthology.orchestra.run.trial168.stderr:1+0 records out 2026-02-22T23:02:31.632 INFO:teuthology.orchestra.run.trial168.stderr:512 bytes copied, 0.000203489 s, 2.5 MB/s 2026-02-22T23:02:31.633 DEBUG:teuthology.orchestra.run.trial168:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2026-02-22T23:02:31.691 DEBUG:teuthology.orchestra.run.trial168:> stat /dev/vg_nvme/lv_3 2026-02-22T23:02:31.748 INFO:teuthology.orchestra.run.trial168.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2026-02-22T23:02:31.748 INFO:teuthology.orchestra.run.trial168.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2026-02-22T23:02:31.748 INFO:teuthology.orchestra.run.trial168.stdout:Device: 6h/6d Inode: 1301 Links: 1 2026-02-22T23:02:31.748 INFO:teuthology.orchestra.run.trial168.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2026-02-22T23:02:31.748 INFO:teuthology.orchestra.run.trial168.stdout:Context: system_u:object_r:device_t:s0 2026-02-22T23:02:31.748 INFO:teuthology.orchestra.run.trial168.stdout:Access: 2026-02-22 23:02:26.611762933 +0000 2026-02-22T23:02:31.749 INFO:teuthology.orchestra.run.trial168.stdout:Modify: 2026-02-22 23:00:32.735411326 +0000 2026-02-22T23:02:31.749 INFO:teuthology.orchestra.run.trial168.stdout:Change: 2026-02-22 23:00:32.735411326 +0000 2026-02-22T23:02:31.749 INFO:teuthology.orchestra.run.trial168.stdout: Birth: 2026-02-22 23:00:32.735411326 +0000 2026-02-22T23:02:31.749 DEBUG:teuthology.orchestra.run.trial168:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2026-02-22T23:02:31.810 INFO:teuthology.orchestra.run.trial168.stderr:1+0 records in 2026-02-22T23:02:31.811 INFO:teuthology.orchestra.run.trial168.stderr:1+0 records out 2026-02-22T23:02:31.811 INFO:teuthology.orchestra.run.trial168.stderr:512 bytes copied, 0.000145631 s, 3.5 MB/s 2026-02-22T23:02:31.812 DEBUG:teuthology.orchestra.run.trial168:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2026-02-22T23:02:31.869 DEBUG:teuthology.orchestra.run.trial168:> stat /dev/vg_nvme/lv_4 2026-02-22T23:02:31.925 INFO:teuthology.orchestra.run.trial168.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2026-02-22T23:02:31.925 INFO:teuthology.orchestra.run.trial168.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2026-02-22T23:02:31.925 INFO:teuthology.orchestra.run.trial168.stdout:Device: 6h/6d Inode: 1300 Links: 1 2026-02-22T23:02:31.925 INFO:teuthology.orchestra.run.trial168.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2026-02-22T23:02:31.925 INFO:teuthology.orchestra.run.trial168.stdout:Context: system_u:object_r:device_t:s0 2026-02-22T23:02:31.925 INFO:teuthology.orchestra.run.trial168.stdout:Access: 2026-02-22 23:02:26.612762923 +0000 2026-02-22T23:02:31.925 INFO:teuthology.orchestra.run.trial168.stdout:Modify: 2026-02-22 23:00:32.735411326 +0000 2026-02-22T23:02:31.925 INFO:teuthology.orchestra.run.trial168.stdout:Change: 2026-02-22 23:00:32.735411326 +0000 2026-02-22T23:02:31.925 INFO:teuthology.orchestra.run.trial168.stdout: Birth: 2026-02-22 23:00:32.735411326 +0000 2026-02-22T23:02:31.926 DEBUG:teuthology.orchestra.run.trial168:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2026-02-22T23:02:31.987 INFO:teuthology.orchestra.run.trial168.stderr:1+0 records in 2026-02-22T23:02:31.987 INFO:teuthology.orchestra.run.trial168.stderr:1+0 records out 2026-02-22T23:02:31.987 INFO:teuthology.orchestra.run.trial168.stderr:512 bytes copied, 0.000146202 s, 3.5 MB/s 2026-02-22T23:02:31.989 DEBUG:teuthology.orchestra.run.trial168:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2026-02-22T23:02:32.048 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'} 2026-02-22T23:02:32.048 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@trial168.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'}} 2026-02-22T23:02:32.048 INFO:tasks.ceph:Generating config... 2026-02-22T23:02:32.051 INFO:tasks.ceph:[global] mon warn on pool no app = False 2026-02-22T23:02:32.051 INFO:tasks.ceph:[mgr] debug mgr = 20 2026-02-22T23:02:32.051 INFO:tasks.ceph:[mgr] debug ms = 1 2026-02-22T23:02:32.051 INFO:tasks.ceph:[mon] debug mon = 20 2026-02-22T23:02:32.051 INFO:tasks.ceph:[mon] debug ms = 1 2026-02-22T23:02:32.051 INFO:tasks.ceph:[mon] debug paxos = 20 2026-02-22T23:02:32.051 INFO:tasks.ceph:[osd] bluefs allocator = hybrid 2026-02-22T23:02:32.051 INFO:tasks.ceph:[osd] bluestore allocator = hybrid 2026-02-22T23:02:32.052 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2026-02-22T23:02:32.052 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2026-02-22T23:02:32.052 INFO:tasks.ceph:[osd] debug bluefs = 1/20 2026-02-22T23:02:32.052 INFO:tasks.ceph:[osd] debug bluestore = 1/20 2026-02-22T23:02:32.052 INFO:tasks.ceph:[osd] debug ms = 1 2026-02-22T23:02:32.052 INFO:tasks.ceph:[osd] debug osd = 20 2026-02-22T23:02:32.052 INFO:tasks.ceph:[osd] debug rocksdb = 4/10 2026-02-22T23:02:32.052 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2026-02-22T23:02:32.052 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2026-02-22T23:02:32.052 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2026-02-22T23:02:32.052 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2026-02-22T23:02:32.052 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2026-02-22T23:02:32.053 INFO:tasks.ceph:[osd] osd shutdown pgref assert = True 2026-02-22T23:02:32.053 INFO:tasks.ceph:Setting up mon.a... 2026-02-22T23:02:32.053 DEBUG:teuthology.orchestra.run.trial168:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2026-02-22T23:02:32.124 INFO:teuthology.orchestra.run.trial168.stdout:creating /etc/ceph/ceph.keyring 2026-02-22T23:02:32.127 DEBUG:teuthology.orchestra.run.trial168:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2026-02-22T23:02:32.204 DEBUG:teuthology.orchestra.run.trial168:> sudo chmod 0644 /etc/ceph/ceph.keyring 2026-02-22T23:02:32.270 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '10.20.193.168')] 2026-02-22T23:02:32.271 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '10.20.193.168', 'mon warn on pool no app': False}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd mclock skip benchmark': 'true', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bluefs allocator': 'hybrid', 'bluestore allocator': 'hybrid', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd shutdown pgref assert': True}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false'}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok'}, 'mon.a': {}} 2026-02-22T23:02:32.272 DEBUG:teuthology.orchestra.run.trial168:> set -ex 2026-02-22T23:02:32.272 DEBUG:teuthology.orchestra.run.trial168:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2026-02-22T23:02:32.326 DEBUG:teuthology.orchestra.run.trial168:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 10.20.193.168 --print /home/ubuntu/cephtest/ceph.monmap 2026-02-22T23:02:32.399 INFO:teuthology.orchestra.run.trial168.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2026-02-22T23:02:32.399 INFO:teuthology.orchestra.run.trial168.stdout:monmaptool: generated fsid 7798f833-50d5-4df5-a39c-9dab1506b7db 2026-02-22T23:02:32.399 INFO:teuthology.orchestra.run.trial168.stdout:setting min_mon_release = quincy 2026-02-22T23:02:32.399 INFO:teuthology.orchestra.run.trial168.stdout:epoch 0 2026-02-22T23:02:32.400 INFO:teuthology.orchestra.run.trial168.stdout:fsid 7798f833-50d5-4df5-a39c-9dab1506b7db 2026-02-22T23:02:32.400 INFO:teuthology.orchestra.run.trial168.stdout:last_changed 2026-02-22T23:02:32.398292+0000 2026-02-22T23:02:32.400 INFO:teuthology.orchestra.run.trial168.stdout:created 2026-02-22T23:02:32.398292+0000 2026-02-22T23:02:32.400 INFO:teuthology.orchestra.run.trial168.stdout:min_mon_release 17 (quincy) 2026-02-22T23:02:32.400 INFO:teuthology.orchestra.run.trial168.stdout:election_strategy: 1 2026-02-22T23:02:32.400 INFO:teuthology.orchestra.run.trial168.stdout:0: [v2:10.20.193.168:3300/0,v1:10.20.193.168:6789/0] mon.a 2026-02-22T23:02:32.400 INFO:teuthology.orchestra.run.trial168.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (1 monitors) 2026-02-22T23:02:32.402 DEBUG:teuthology.orchestra.run.trial168:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2026-02-22T23:02:32.458 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 7798f833-50d5-4df5-a39c-9dab1506b7db... 2026-02-22T23:02:32.459 DEBUG:teuthology.orchestra.run.trial168:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2026-02-22T23:02:32.537 INFO:teuthology.orchestra.run.trial168.stdout:[global] 2026-02-22T23:02:32.537 INFO:teuthology.orchestra.run.trial168.stdout: chdir = "" 2026-02-22T23:02:32.537 INFO:teuthology.orchestra.run.trial168.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2026-02-22T23:02:32.537 INFO:teuthology.orchestra.run.trial168.stdout: auth supported = cephx 2026-02-22T23:02:32.537 INFO:teuthology.orchestra.run.trial168.stdout: 2026-02-22T23:02:32.537 INFO:teuthology.orchestra.run.trial168.stdout: filestore xattr use omap = true 2026-02-22T23:02:32.537 INFO:teuthology.orchestra.run.trial168.stdout: 2026-02-22T23:02:32.537 INFO:teuthology.orchestra.run.trial168.stdout: mon clock drift allowed = 1.000 2026-02-22T23:02:32.537 INFO:teuthology.orchestra.run.trial168.stdout: 2026-02-22T23:02:32.537 INFO:teuthology.orchestra.run.trial168.stdout: osd crush chooseleaf type = 0 2026-02-22T23:02:32.537 INFO:teuthology.orchestra.run.trial168.stdout: auth debug = true 2026-02-22T23:02:32.537 INFO:teuthology.orchestra.run.trial168.stdout: 2026-02-22T23:02:32.537 INFO:teuthology.orchestra.run.trial168.stdout: ms die on old message = true 2026-02-22T23:02:32.537 INFO:teuthology.orchestra.run.trial168.stdout: ms die on bug = true 2026-02-22T23:02:32.537 INFO:teuthology.orchestra.run.trial168.stdout: 2026-02-22T23:02:32.537 INFO:teuthology.orchestra.run.trial168.stdout: mon max pg per osd = 10000 # >= luminous 2026-02-22T23:02:32.538 INFO:teuthology.orchestra.run.trial168.stdout: mon pg warn max object skew = 0 2026-02-22T23:02:32.538 INFO:teuthology.orchestra.run.trial168.stdout: 2026-02-22T23:02:32.538 INFO:teuthology.orchestra.run.trial168.stdout: # disable pg_autoscaler by default for new pools 2026-02-22T23:02:32.538 INFO:teuthology.orchestra.run.trial168.stdout: osd_pool_default_pg_autoscale_mode = off 2026-02-22T23:02:32.538 INFO:teuthology.orchestra.run.trial168.stdout: 2026-02-22T23:02:32.538 INFO:teuthology.orchestra.run.trial168.stdout: osd pool default size = 2 2026-02-22T23:02:32.538 INFO:teuthology.orchestra.run.trial168.stdout: 2026-02-22T23:02:32.538 INFO:teuthology.orchestra.run.trial168.stdout: mon osd allow primary affinity = true 2026-02-22T23:02:32.538 INFO:teuthology.orchestra.run.trial168.stdout: mon osd allow pg remap = true 2026-02-22T23:02:32.538 INFO:teuthology.orchestra.run.trial168.stdout: mon warn on legacy crush tunables = false 2026-02-22T23:02:32.538 INFO:teuthology.orchestra.run.trial168.stdout: mon warn on crush straw calc version zero = false 2026-02-22T23:02:32.538 INFO:teuthology.orchestra.run.trial168.stdout: mon warn on no sortbitwise = false 2026-02-22T23:02:32.538 INFO:teuthology.orchestra.run.trial168.stdout: mon warn on osd down out interval zero = false 2026-02-22T23:02:32.538 INFO:teuthology.orchestra.run.trial168.stdout: mon warn on too few osds = false 2026-02-22T23:02:32.538 INFO:teuthology.orchestra.run.trial168.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2026-02-22T23:02:32.538 INFO:teuthology.orchestra.run.trial168.stdout: mon_warn_on_pool_no_redundancy = false 2026-02-22T23:02:32.538 INFO:teuthology.orchestra.run.trial168.stdout: mon_allow_pool_size_one = true 2026-02-22T23:02:32.538 INFO:teuthology.orchestra.run.trial168.stdout: 2026-02-22T23:02:32.538 INFO:teuthology.orchestra.run.trial168.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2026-02-22T23:02:32.538 INFO:teuthology.orchestra.run.trial168.stdout: 2026-02-22T23:02:32.538 INFO:teuthology.orchestra.run.trial168.stdout: osd default data pool replay window = 5 2026-02-22T23:02:32.538 INFO:teuthology.orchestra.run.trial168.stdout: 2026-02-22T23:02:32.538 INFO:teuthology.orchestra.run.trial168.stdout: mon allow pool delete = true 2026-02-22T23:02:32.538 INFO:teuthology.orchestra.run.trial168.stdout: 2026-02-22T23:02:32.538 INFO:teuthology.orchestra.run.trial168.stdout: mon cluster log file level = debug 2026-02-22T23:02:32.538 INFO:teuthology.orchestra.run.trial168.stdout: debug asserts on shutdown = true 2026-02-22T23:02:32.538 INFO:teuthology.orchestra.run.trial168.stdout: mon health detail to clog = false 2026-02-22T23:02:32.538 INFO:teuthology.orchestra.run.trial168.stdout: mon host = 10.20.193.168 2026-02-22T23:02:32.538 INFO:teuthology.orchestra.run.trial168.stdout: mon warn on pool no app = False 2026-02-22T23:02:32.538 INFO:teuthology.orchestra.run.trial168.stdout: fsid = 7798f833-50d5-4df5-a39c-9dab1506b7db 2026-02-22T23:02:32.539 INFO:teuthology.orchestra.run.trial168.stdout: 2026-02-22T23:02:32.539 INFO:teuthology.orchestra.run.trial168.stdout:[osd] 2026-02-22T23:02:32.539 INFO:teuthology.orchestra.run.trial168.stdout: osd journal size = 100 2026-02-22T23:02:32.539 INFO:teuthology.orchestra.run.trial168.stdout: 2026-02-22T23:02:32.539 INFO:teuthology.orchestra.run.trial168.stdout: osd scrub load threshold = 5.0 2026-02-22T23:02:32.539 INFO:teuthology.orchestra.run.trial168.stdout: osd scrub max interval = 600 2026-02-22T23:02:32.539 INFO:teuthology.orchestra.run.trial168.stdout: osd mclock profile = high_recovery_ops 2026-02-22T23:02:32.539 INFO:teuthology.orchestra.run.trial168.stdout: osd mclock skip benchmark = true 2026-02-22T23:02:32.539 INFO:teuthology.orchestra.run.trial168.stdout: 2026-02-22T23:02:32.539 INFO:teuthology.orchestra.run.trial168.stdout: osd recover clone overlap = true 2026-02-22T23:02:32.539 INFO:teuthology.orchestra.run.trial168.stdout: osd recovery max chunk = 1048576 2026-02-22T23:02:32.539 INFO:teuthology.orchestra.run.trial168.stdout: 2026-02-22T23:02:32.539 INFO:teuthology.orchestra.run.trial168.stdout: osd debug shutdown = true 2026-02-22T23:02:32.540 INFO:teuthology.orchestra.run.trial168.stdout: osd debug op order = true 2026-02-22T23:02:32.540 INFO:teuthology.orchestra.run.trial168.stdout: osd debug verify stray on activate = true 2026-02-22T23:02:32.540 INFO:teuthology.orchestra.run.trial168.stdout: osd debug trim objects = true 2026-02-22T23:02:32.540 INFO:teuthology.orchestra.run.trial168.stdout: 2026-02-22T23:02:32.540 INFO:teuthology.orchestra.run.trial168.stdout: osd open classes on start = true 2026-02-22T23:02:32.540 INFO:teuthology.orchestra.run.trial168.stdout: osd debug pg log writeout = true 2026-02-22T23:02:32.540 INFO:teuthology.orchestra.run.trial168.stdout: 2026-02-22T23:02:32.540 INFO:teuthology.orchestra.run.trial168.stdout: osd deep scrub update digest min age = 30 2026-02-22T23:02:32.540 INFO:teuthology.orchestra.run.trial168.stdout: 2026-02-22T23:02:32.540 INFO:teuthology.orchestra.run.trial168.stdout: osd map max advance = 10 2026-02-22T23:02:32.540 INFO:teuthology.orchestra.run.trial168.stdout: 2026-02-22T23:02:32.540 INFO:teuthology.orchestra.run.trial168.stdout: journal zero on create = true 2026-02-22T23:02:32.540 INFO:teuthology.orchestra.run.trial168.stdout: 2026-02-22T23:02:32.541 INFO:teuthology.orchestra.run.trial168.stdout: filestore ondisk finisher threads = 3 2026-02-22T23:02:32.541 INFO:teuthology.orchestra.run.trial168.stdout: filestore apply finisher threads = 3 2026-02-22T23:02:32.541 INFO:teuthology.orchestra.run.trial168.stdout: 2026-02-22T23:02:32.541 INFO:teuthology.orchestra.run.trial168.stdout: bdev debug aio = true 2026-02-22T23:02:32.541 INFO:teuthology.orchestra.run.trial168.stdout: osd debug misdirected ops = true 2026-02-22T23:02:32.541 INFO:teuthology.orchestra.run.trial168.stdout: bluefs allocator = hybrid 2026-02-22T23:02:32.541 INFO:teuthology.orchestra.run.trial168.stdout: bluestore allocator = hybrid 2026-02-22T23:02:32.541 INFO:teuthology.orchestra.run.trial168.stdout: bluestore block size = 96636764160 2026-02-22T23:02:32.541 INFO:teuthology.orchestra.run.trial168.stdout: bluestore fsck on mount = True 2026-02-22T23:02:32.541 INFO:teuthology.orchestra.run.trial168.stdout: debug bluefs = 1/20 2026-02-22T23:02:32.541 INFO:teuthology.orchestra.run.trial168.stdout: debug bluestore = 1/20 2026-02-22T23:02:32.541 INFO:teuthology.orchestra.run.trial168.stdout: debug ms = 1 2026-02-22T23:02:32.541 INFO:teuthology.orchestra.run.trial168.stdout: debug osd = 20 2026-02-22T23:02:32.542 INFO:teuthology.orchestra.run.trial168.stdout: debug rocksdb = 4/10 2026-02-22T23:02:32.542 INFO:teuthology.orchestra.run.trial168.stdout: mon osd backfillfull_ratio = 0.85 2026-02-22T23:02:32.542 INFO:teuthology.orchestra.run.trial168.stdout: mon osd full ratio = 0.9 2026-02-22T23:02:32.542 INFO:teuthology.orchestra.run.trial168.stdout: mon osd nearfull ratio = 0.8 2026-02-22T23:02:32.542 INFO:teuthology.orchestra.run.trial168.stdout: osd failsafe full ratio = 0.95 2026-02-22T23:02:32.542 INFO:teuthology.orchestra.run.trial168.stdout: osd objectstore = bluestore 2026-02-22T23:02:32.542 INFO:teuthology.orchestra.run.trial168.stdout: osd shutdown pgref assert = True 2026-02-22T23:02:32.542 INFO:teuthology.orchestra.run.trial168.stdout: 2026-02-22T23:02:32.542 INFO:teuthology.orchestra.run.trial168.stdout:[mgr] 2026-02-22T23:02:32.542 INFO:teuthology.orchestra.run.trial168.stdout: debug ms = 1 2026-02-22T23:02:32.542 INFO:teuthology.orchestra.run.trial168.stdout: debug mgr = 20 2026-02-22T23:02:32.542 INFO:teuthology.orchestra.run.trial168.stdout: debug mon = 20 2026-02-22T23:02:32.543 INFO:teuthology.orchestra.run.trial168.stdout: debug auth = 20 2026-02-22T23:02:32.543 INFO:teuthology.orchestra.run.trial168.stdout: mon reweight min pgs per osd = 4 2026-02-22T23:02:32.543 INFO:teuthology.orchestra.run.trial168.stdout: mon reweight min bytes per osd = 10 2026-02-22T23:02:32.543 INFO:teuthology.orchestra.run.trial168.stdout: mgr/telemetry/nag = false 2026-02-22T23:02:32.543 INFO:teuthology.orchestra.run.trial168.stdout: 2026-02-22T23:02:32.543 INFO:teuthology.orchestra.run.trial168.stdout:[mon] 2026-02-22T23:02:32.543 INFO:teuthology.orchestra.run.trial168.stdout: debug ms = 1 2026-02-22T23:02:32.543 INFO:teuthology.orchestra.run.trial168.stdout: debug mon = 20 2026-02-22T23:02:32.543 INFO:teuthology.orchestra.run.trial168.stdout: debug paxos = 20 2026-02-22T23:02:32.543 INFO:teuthology.orchestra.run.trial168.stdout: debug auth = 20 2026-02-22T23:02:32.543 INFO:teuthology.orchestra.run.trial168.stdout: mon data avail warn = 5 2026-02-22T23:02:32.543 INFO:teuthology.orchestra.run.trial168.stdout: mon mgr mkfs grace = 240 2026-02-22T23:02:32.543 INFO:teuthology.orchestra.run.trial168.stdout: mon reweight min pgs per osd = 4 2026-02-22T23:02:32.544 INFO:teuthology.orchestra.run.trial168.stdout: mon osd reporter subtree level = osd 2026-02-22T23:02:32.544 INFO:teuthology.orchestra.run.trial168.stdout: mon osd prime pg temp = true 2026-02-22T23:02:32.544 INFO:teuthology.orchestra.run.trial168.stdout: mon reweight min bytes per osd = 10 2026-02-22T23:02:32.544 INFO:teuthology.orchestra.run.trial168.stdout: 2026-02-22T23:02:32.544 INFO:teuthology.orchestra.run.trial168.stdout: # rotate auth tickets quickly to exercise renewal paths 2026-02-22T23:02:32.544 INFO:teuthology.orchestra.run.trial168.stdout: auth mon ticket ttl = 660 # 11m 2026-02-22T23:02:32.544 INFO:teuthology.orchestra.run.trial168.stdout: auth service ticket ttl = 240 # 4m 2026-02-22T23:02:32.544 INFO:teuthology.orchestra.run.trial168.stdout: 2026-02-22T23:02:32.544 INFO:teuthology.orchestra.run.trial168.stdout: # don't complain about insecure global_id in the test suite 2026-02-22T23:02:32.544 INFO:teuthology.orchestra.run.trial168.stdout: mon_warn_on_insecure_global_id_reclaim = false 2026-02-22T23:02:32.544 INFO:teuthology.orchestra.run.trial168.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2026-02-22T23:02:32.544 INFO:teuthology.orchestra.run.trial168.stdout: 2026-02-22T23:02:32.544 INFO:teuthology.orchestra.run.trial168.stdout: # 1m isn't quite enough 2026-02-22T23:02:32.544 INFO:teuthology.orchestra.run.trial168.stdout: mon_down_mkfs_grace = 2m 2026-02-22T23:02:32.545 INFO:teuthology.orchestra.run.trial168.stdout: 2026-02-22T23:02:32.545 INFO:teuthology.orchestra.run.trial168.stdout: mon_warn_on_filestore_osds = false 2026-02-22T23:02:32.545 INFO:teuthology.orchestra.run.trial168.stdout: 2026-02-22T23:02:32.545 INFO:teuthology.orchestra.run.trial168.stdout:[client] 2026-02-22T23:02:32.545 INFO:teuthology.orchestra.run.trial168.stdout: rgw cache enabled = true 2026-02-22T23:02:32.545 INFO:teuthology.orchestra.run.trial168.stdout: rgw enable ops log = true 2026-02-22T23:02:32.545 INFO:teuthology.orchestra.run.trial168.stdout: rgw enable usage log = true 2026-02-22T23:02:32.545 INFO:teuthology.orchestra.run.trial168.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2026-02-22T23:02:32.545 INFO:teuthology.orchestra.run.trial168.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2026-02-22T23:02:32.545 INFO:teuthology.orchestra.run.trial168.stdout:[mon.a] 2026-02-22T23:02:32.547 INFO:tasks.ceph:Creating admin key on mon.a... 2026-02-22T23:02:32.547 DEBUG:teuthology.orchestra.run.trial168:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2026-02-22T23:02:32.628 INFO:tasks.ceph:Copying monmap to all nodes... 2026-02-22T23:02:32.628 DEBUG:teuthology.orchestra.run.trial168:> set -ex 2026-02-22T23:02:32.628 DEBUG:teuthology.orchestra.run.trial168:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2026-02-22T23:02:32.683 DEBUG:teuthology.orchestra.run.trial168:> set -ex 2026-02-22T23:02:32.683 DEBUG:teuthology.orchestra.run.trial168:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2026-02-22T23:02:32.740 INFO:tasks.ceph:Sending monmap to node ubuntu@trial168.front.sepia.ceph.com 2026-02-22T23:02:32.741 DEBUG:teuthology.orchestra.run.trial168:> set -ex 2026-02-22T23:02:32.741 DEBUG:teuthology.orchestra.run.trial168:> sudo dd of=/etc/ceph/ceph.keyring 2026-02-22T23:02:32.741 DEBUG:teuthology.orchestra.run.trial168:> sudo chmod 0644 /etc/ceph/ceph.keyring 2026-02-22T23:02:32.814 DEBUG:teuthology.orchestra.run.trial168:> set -ex 2026-02-22T23:02:32.814 DEBUG:teuthology.orchestra.run.trial168:> dd of=/home/ubuntu/cephtest/ceph.monmap 2026-02-22T23:02:32.871 INFO:tasks.ceph:Setting up mon nodes... 2026-02-22T23:02:32.872 INFO:tasks.ceph:Setting up mgr nodes... 2026-02-22T23:02:32.872 DEBUG:teuthology.orchestra.run.trial168:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2026-02-22T23:02:32.953 INFO:teuthology.orchestra.run.trial168.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2026-02-22T23:02:32.956 INFO:tasks.ceph:Setting up mds nodes... 2026-02-22T23:02:32.957 INFO:tasks.ceph_client:Setting up client nodes... 2026-02-22T23:02:32.957 DEBUG:teuthology.orchestra.run.trial168:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2026-02-22T23:02:32.991 INFO:teuthology.orchestra.run.trial168.stdout:creating /etc/ceph/ceph.client.0.keyring 2026-02-22T23:02:33.002 INFO:tasks.ceph:Running mkfs on osd nodes... 2026-02-22T23:02:33.002 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@trial168.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'}} 2026-02-22T23:02:33.002 DEBUG:teuthology.orchestra.run.trial168:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2026-02-22T23:02:33.066 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'} 2026-02-22T23:02:33.066 INFO:tasks.ceph:role: osd.0 2026-02-22T23:02:33.067 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@trial168.front.sepia.ceph.com 2026-02-22T23:02:33.067 DEBUG:teuthology.orchestra.run.trial168:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2026-02-22T23:02:33.134 INFO:teuthology.orchestra.run.trial168.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=45875200 blks 2026-02-22T23:02:33.135 INFO:teuthology.orchestra.run.trial168.stdout: = sectsz=4096 attr=2, projid32bit=1 2026-02-22T23:02:33.135 INFO:teuthology.orchestra.run.trial168.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2026-02-22T23:02:33.135 INFO:teuthology.orchestra.run.trial168.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2026-02-22T23:02:33.135 INFO:teuthology.orchestra.run.trial168.stdout:data = bsize=4096 blocks=183500800, imaxpct=25 2026-02-22T23:02:33.135 INFO:teuthology.orchestra.run.trial168.stdout: = sunit=0 swidth=0 blks 2026-02-22T23:02:33.135 INFO:teuthology.orchestra.run.trial168.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2026-02-22T23:02:33.135 INFO:teuthology.orchestra.run.trial168.stdout:log =internal log bsize=4096 blocks=89600, version=2 2026-02-22T23:02:33.136 INFO:teuthology.orchestra.run.trial168.stdout: = sectsz=4096 sunit=1 blks, lazy-count=1 2026-02-22T23:02:33.136 INFO:teuthology.orchestra.run.trial168.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2026-02-22T23:02:33.703 INFO:teuthology.orchestra.run.trial168.stdout:Discarding blocks...Done. 2026-02-22T23:02:33.706 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@trial168.front.sepia.ceph.com -o noatime 2026-02-22T23:02:33.706 DEBUG:teuthology.orchestra.run.trial168:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2026-02-22T23:02:33.774 DEBUG:teuthology.orchestra.run.trial168:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2026-02-22T23:02:33.841 DEBUG:teuthology.orchestra.run.trial168:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2026-02-22T23:02:33.904 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'} 2026-02-22T23:02:33.904 INFO:tasks.ceph:role: osd.1 2026-02-22T23:02:33.904 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@trial168.front.sepia.ceph.com 2026-02-22T23:02:33.904 DEBUG:teuthology.orchestra.run.trial168:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2026-02-22T23:02:33.971 INFO:teuthology.orchestra.run.trial168.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=45875200 blks 2026-02-22T23:02:33.971 INFO:teuthology.orchestra.run.trial168.stdout: = sectsz=4096 attr=2, projid32bit=1 2026-02-22T23:02:33.971 INFO:teuthology.orchestra.run.trial168.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2026-02-22T23:02:33.971 INFO:teuthology.orchestra.run.trial168.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2026-02-22T23:02:33.971 INFO:teuthology.orchestra.run.trial168.stdout:data = bsize=4096 blocks=183500800, imaxpct=25 2026-02-22T23:02:33.971 INFO:teuthology.orchestra.run.trial168.stdout: = sunit=0 swidth=0 blks 2026-02-22T23:02:33.972 INFO:teuthology.orchestra.run.trial168.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2026-02-22T23:02:33.972 INFO:teuthology.orchestra.run.trial168.stdout:log =internal log bsize=4096 blocks=89600, version=2 2026-02-22T23:02:33.972 INFO:teuthology.orchestra.run.trial168.stdout: = sectsz=4096 sunit=1 blks, lazy-count=1 2026-02-22T23:02:33.972 INFO:teuthology.orchestra.run.trial168.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2026-02-22T23:02:34.530 INFO:teuthology.orchestra.run.trial168.stdout:Discarding blocks...Done. 2026-02-22T23:02:34.532 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@trial168.front.sepia.ceph.com -o noatime 2026-02-22T23:02:34.533 DEBUG:teuthology.orchestra.run.trial168:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2026-02-22T23:02:34.600 DEBUG:teuthology.orchestra.run.trial168:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2026-02-22T23:02:34.667 DEBUG:teuthology.orchestra.run.trial168:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2026-02-22T23:02:34.730 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'} 2026-02-22T23:02:34.730 INFO:tasks.ceph:role: osd.2 2026-02-22T23:02:34.730 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@trial168.front.sepia.ceph.com 2026-02-22T23:02:34.730 DEBUG:teuthology.orchestra.run.trial168:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2026-02-22T23:02:34.795 INFO:teuthology.orchestra.run.trial168.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=45875200 blks 2026-02-22T23:02:34.795 INFO:teuthology.orchestra.run.trial168.stdout: = sectsz=4096 attr=2, projid32bit=1 2026-02-22T23:02:34.795 INFO:teuthology.orchestra.run.trial168.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2026-02-22T23:02:34.795 INFO:teuthology.orchestra.run.trial168.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2026-02-22T23:02:34.795 INFO:teuthology.orchestra.run.trial168.stdout:data = bsize=4096 blocks=183500800, imaxpct=25 2026-02-22T23:02:34.795 INFO:teuthology.orchestra.run.trial168.stdout: = sunit=0 swidth=0 blks 2026-02-22T23:02:34.795 INFO:teuthology.orchestra.run.trial168.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2026-02-22T23:02:34.795 INFO:teuthology.orchestra.run.trial168.stdout:log =internal log bsize=4096 blocks=89600, version=2 2026-02-22T23:02:34.795 INFO:teuthology.orchestra.run.trial168.stdout: = sectsz=4096 sunit=1 blks, lazy-count=1 2026-02-22T23:02:34.795 INFO:teuthology.orchestra.run.trial168.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2026-02-22T23:02:35.351 INFO:teuthology.orchestra.run.trial168.stdout:Discarding blocks...Done. 2026-02-22T23:02:35.353 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@trial168.front.sepia.ceph.com -o noatime 2026-02-22T23:02:35.353 DEBUG:teuthology.orchestra.run.trial168:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2026-02-22T23:02:35.421 DEBUG:teuthology.orchestra.run.trial168:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2026-02-22T23:02:35.487 DEBUG:teuthology.orchestra.run.trial168:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2026-02-22T23:02:35.566 INFO:teuthology.orchestra.run.trial168.stderr:2026-02-22T23:02:35.563+0000 7f7b5461a780 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2026-02-22T23:02:35.566 INFO:teuthology.orchestra.run.trial168.stderr:2026-02-22T23:02:35.563+0000 7f7b5461a780 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2026-02-22T23:02:35.566 INFO:teuthology.orchestra.run.trial168.stderr:2026-02-22T23:02:35.563+0000 7f7b5461a780 -1 bdev(0x5613fbde1800 /var/lib/ceph/osd/ceph-0/block) open stat got: (1) Operation not permitted 2026-02-22T23:02:35.566 INFO:teuthology.orchestra.run.trial168.stderr:2026-02-22T23:02:35.563+0000 7f7b5461a780 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2026-02-22T23:02:38.712 DEBUG:teuthology.orchestra.run.trial168:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2026-02-22T23:02:38.776 DEBUG:teuthology.orchestra.run.trial168:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2026-02-22T23:02:38.855 INFO:teuthology.orchestra.run.trial168.stderr:2026-02-22T23:02:38.853+0000 7f1a0a1f5780 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2026-02-22T23:02:38.855 INFO:teuthology.orchestra.run.trial168.stderr:2026-02-22T23:02:38.853+0000 7f1a0a1f5780 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2026-02-22T23:02:38.855 INFO:teuthology.orchestra.run.trial168.stderr:2026-02-22T23:02:38.853+0000 7f1a0a1f5780 -1 bdev(0x55af614d1800 /var/lib/ceph/osd/ceph-1/block) open stat got: (1) Operation not permitted 2026-02-22T23:02:38.855 INFO:teuthology.orchestra.run.trial168.stderr:2026-02-22T23:02:38.853+0000 7f1a0a1f5780 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2026-02-22T23:02:41.762 DEBUG:teuthology.orchestra.run.trial168:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2026-02-22T23:02:41.827 DEBUG:teuthology.orchestra.run.trial168:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2026-02-22T23:02:41.904 INFO:teuthology.orchestra.run.trial168.stderr:2026-02-22T23:02:41.902+0000 7fb1fce2c780 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2026-02-22T23:02:41.904 INFO:teuthology.orchestra.run.trial168.stderr:2026-02-22T23:02:41.902+0000 7fb1fce2c780 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2026-02-22T23:02:41.904 INFO:teuthology.orchestra.run.trial168.stderr:2026-02-22T23:02:41.902+0000 7fb1fce2c780 -1 bdev(0x55814abf7800 /var/lib/ceph/osd/ceph-2/block) open stat got: (1) Operation not permitted 2026-02-22T23:02:41.904 INFO:teuthology.orchestra.run.trial168.stderr:2026-02-22T23:02:41.902+0000 7fb1fce2c780 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2026-02-22T23:02:45.061 DEBUG:teuthology.orchestra.run.trial168:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2026-02-22T23:02:45.128 INFO:tasks.ceph:Reading keys from all nodes... 2026-02-22T23:02:45.128 DEBUG:teuthology.orchestra.run.trial168:> set -ex 2026-02-22T23:02:45.129 DEBUG:teuthology.orchestra.run.trial168:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2026-02-22T23:02:45.192 DEBUG:teuthology.orchestra.run.trial168:> set -ex 2026-02-22T23:02:45.192 DEBUG:teuthology.orchestra.run.trial168:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2026-02-22T23:02:45.255 DEBUG:teuthology.orchestra.run.trial168:> set -ex 2026-02-22T23:02:45.256 DEBUG:teuthology.orchestra.run.trial168:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2026-02-22T23:02:45.319 DEBUG:teuthology.orchestra.run.trial168:> set -ex 2026-02-22T23:02:45.319 DEBUG:teuthology.orchestra.run.trial168:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2026-02-22T23:02:45.384 DEBUG:teuthology.orchestra.run.trial168:> set -ex 2026-02-22T23:02:45.385 DEBUG:teuthology.orchestra.run.trial168:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2026-02-22T23:02:45.440 INFO:tasks.ceph:Adding keys to all mons... 2026-02-22T23:02:45.440 DEBUG:teuthology.orchestra.run.trial168:> sudo tee -a /etc/ceph/ceph.keyring 2026-02-22T23:02:45.502 INFO:teuthology.orchestra.run.trial168.stdout:[mgr.x] 2026-02-22T23:02:45.502 INFO:teuthology.orchestra.run.trial168.stdout: key = AQAIi5tpX5rUOBAAgX5opzE2ySpkLnNy1ZfwKw== 2026-02-22T23:02:45.502 INFO:teuthology.orchestra.run.trial168.stdout:[osd.0] 2026-02-22T23:02:45.502 INFO:teuthology.orchestra.run.trial168.stdout: key = AQALi5tptyKrIRAA/mtvm+2ZLF9+4N7z0wTx2A== 2026-02-22T23:02:45.502 INFO:teuthology.orchestra.run.trial168.stdout:[osd.1] 2026-02-22T23:02:45.502 INFO:teuthology.orchestra.run.trial168.stdout: key = AQAOi5tp5mHxMhAAoKX6CY6xLwCi7rVlmJhfUQ== 2026-02-22T23:02:45.502 INFO:teuthology.orchestra.run.trial168.stdout:[osd.2] 2026-02-22T23:02:45.502 INFO:teuthology.orchestra.run.trial168.stdout: key = AQARi5tp1rXbNRAA/DTtndFnB9lHRLkudzyYmA== 2026-02-22T23:02:45.502 INFO:teuthology.orchestra.run.trial168.stdout:[client.0] 2026-02-22T23:02:45.502 INFO:teuthology.orchestra.run.trial168.stdout: key = AQAIi5tp4AsMOxAAxIMemEGfp/MmQgWTErwcAQ== 2026-02-22T23:02:45.503 DEBUG:teuthology.orchestra.run.trial168:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2026-02-22T23:02:45.582 DEBUG:teuthology.orchestra.run.trial168:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-22T23:02:45.662 DEBUG:teuthology.orchestra.run.trial168:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-22T23:02:45.742 DEBUG:teuthology.orchestra.run.trial168:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-22T23:02:45.823 DEBUG:teuthology.orchestra.run.trial168:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2026-02-22T23:02:45.906 INFO:tasks.ceph:Running mkfs on mon nodes... 2026-02-22T23:02:45.906 DEBUG:teuthology.orchestra.run.trial168:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2026-02-22T23:02:45.970 DEBUG:teuthology.orchestra.run.trial168:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2026-02-22T23:02:46.050 DEBUG:teuthology.orchestra.run.trial168:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2026-02-22T23:02:46.115 DEBUG:teuthology.orchestra.run.trial168:> rm -- /home/ubuntu/cephtest/ceph.monmap 2026-02-22T23:02:46.171 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2026-02-22T23:02:46.172 INFO:tasks.ceph.mon.a:Restarting daemon 2026-02-22T23:02:46.172 DEBUG:teuthology.orchestra.run.trial168:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2026-02-22T23:02:46.213 INFO:tasks.ceph.mon.a:Started 2026-02-22T23:02:46.213 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2026-02-22T23:02:46.213 INFO:tasks.ceph.mgr.x:Restarting daemon 2026-02-22T23:02:46.213 DEBUG:teuthology.orchestra.run.trial168:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2026-02-22T23:02:46.215 INFO:tasks.ceph.mgr.x:Started 2026-02-22T23:02:46.215 INFO:tasks.ceph:Setting crush tunables to default 2026-02-22T23:02:46.215 DEBUG:teuthology.orchestra.run.trial168:> sudo ceph --cluster ceph osd crush tunables default 2026-02-22T23:02:46.316 INFO:teuthology.orchestra.run.trial168.stderr:adjusted tunables profile to default 2026-02-22T23:02:46.323 INFO:tasks.ceph:check_enable_crimson: False 2026-02-22T23:02:46.323 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2026-02-22T23:02:46.323 DEBUG:teuthology.orchestra.run.trial168:> set -ex 2026-02-22T23:02:46.323 DEBUG:teuthology.orchestra.run.trial168:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2026-02-22T23:02:46.348 DEBUG:teuthology.orchestra.run.trial168:> set -ex 2026-02-22T23:02:46.348 DEBUG:teuthology.orchestra.run.trial168:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2026-02-22T23:02:46.412 DEBUG:teuthology.orchestra.run.trial168:> set -ex 2026-02-22T23:02:46.413 DEBUG:teuthology.orchestra.run.trial168:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2026-02-22T23:02:46.476 DEBUG:teuthology.orchestra.run.trial168:> sudo ceph --cluster ceph osd new 882c47f4-e016-46e9-b67e-b2d6389fc9f8 0 2026-02-22T23:02:46.618 INFO:teuthology.orchestra.run.trial168.stdout:0 2026-02-22T23:02:46.626 DEBUG:teuthology.orchestra.run.trial168:> sudo ceph --cluster ceph osd new 9835b2d4-f84f-4f74-b135-663786caa176 1 2026-02-22T23:02:46.728 INFO:teuthology.orchestra.run.trial168.stdout:1 2026-02-22T23:02:46.737 DEBUG:teuthology.orchestra.run.trial168:> sudo ceph --cluster ceph osd new 59a03d64-5286-4376-b7d5-80007120931a 2 2026-02-22T23:02:46.830 INFO:tasks.ceph.mgr.x.trial168.stderr:2026-02-22T23:02:46.828+0000 7f07a1b62140 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2026-02-22T23:02:46.838 INFO:teuthology.orchestra.run.trial168.stdout:2 2026-02-22T23:02:46.846 INFO:tasks.ceph.osd.0:Restarting daemon 2026-02-22T23:02:46.846 DEBUG:teuthology.orchestra.run.trial168:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2026-02-22T23:02:46.848 INFO:tasks.ceph.osd.0:Started 2026-02-22T23:02:46.848 INFO:tasks.ceph.osd.1:Restarting daemon 2026-02-22T23:02:46.848 DEBUG:teuthology.orchestra.run.trial168:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2026-02-22T23:02:46.851 INFO:tasks.ceph.osd.1:Started 2026-02-22T23:02:46.851 INFO:tasks.ceph.osd.2:Restarting daemon 2026-02-22T23:02:46.851 DEBUG:teuthology.orchestra.run.trial168:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2026-02-22T23:02:46.852 INFO:tasks.ceph.osd.2:Started 2026-02-22T23:02:46.853 DEBUG:teuthology.orchestra.run.trial168:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2026-02-22T23:02:46.955 INFO:teuthology.orchestra.run.trial168.stdout: 2026-02-22T23:02:46.955 INFO:teuthology.orchestra.run.trial168.stdout:{"epoch":5,"fsid":"7798f833-50d5-4df5-a39c-9dab1506b7db","created":"2026-02-22T23:02:46.276170+0000","modified":"2026-02-22T23:02:46.837523+0000","last_up_change":"0.000000","last_in_change":"2026-02-22T23:02:46.837523+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"882c47f4-e016-46e9-b67e-b2d6389fc9f8","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"9835b2d4-f84f-4f74-b135-663786caa176","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"59a03d64-5286-4376-b7d5-80007120931a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2026-02-22T23:02:46.961 INFO:tasks.ceph.ceph_manager.ceph:[] 2026-02-22T23:02:46.962 INFO:tasks.ceph:Waiting for OSDs to come up 2026-02-22T23:02:46.963 INFO:tasks.ceph.mgr.x.trial168.stderr:2026-02-22T23:02:46.959+0000 7f07a1b62140 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2026-02-22T23:02:46.990 INFO:tasks.ceph.mgr.x.trial168.stderr:2026-02-22T23:02:46.988+0000 7f07a1b62140 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2026-02-22T23:02:47.024 INFO:tasks.ceph.mgr.x.trial168.stderr:2026-02-22T23:02:47.021+0000 7f07a1b62140 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2026-02-22T23:02:47.047 INFO:tasks.ceph.mgr.x.trial168.stderr:2026-02-22T23:02:47.045+0000 7f07a1b62140 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2026-02-22T23:02:47.074 INFO:tasks.ceph.mgr.x.trial168.stderr:2026-02-22T23:02:47.071+0000 7f07a1b62140 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2026-02-22T23:02:47.131 INFO:tasks.ceph.mgr.x.trial168.stderr:2026-02-22T23:02:47.128+0000 7f07a1b62140 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2026-02-22T23:02:47.193 INFO:tasks.ceph.mgr.x.trial168.stderr:/usr/lib64/python3.9/site-packages/scipy/__init__.py:73: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2026-02-22T23:02:47.193 INFO:tasks.ceph.mgr.x.trial168.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2026-02-22T23:02:47.193 INFO:tasks.ceph.mgr.x.trial168.stderr: from numpy import show_config as show_numpy_config 2026-02-22T23:02:47.196 INFO:tasks.ceph.mgr.x.trial168.stderr:2026-02-22T23:02:47.194+0000 7f07a1b62140 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2026-02-22T23:02:47.222 INFO:tasks.ceph.mgr.x.trial168.stderr:2026-02-22T23:02:47.220+0000 7f07a1b62140 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2026-02-22T23:02:47.264 DEBUG:teuthology.orchestra.run.trial168:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2026-02-22T23:02:47.278 INFO:tasks.ceph.mgr.x.trial168.stderr:2026-02-22T23:02:47.276+0000 7f07a1b62140 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2026-02-22T23:02:47.323 INFO:tasks.ceph.mgr.x.trial168.stderr:2026-02-22T23:02:47.321+0000 7f07a1b62140 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2026-02-22T23:02:47.350 INFO:tasks.ceph.mgr.x.trial168.stderr:2026-02-22T23:02:47.347+0000 7f07a1b62140 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2026-02-22T23:02:47.356 INFO:teuthology.misc.health.trial168.stdout: 2026-02-22T23:02:47.356 INFO:teuthology.misc.health.trial168.stdout:{"epoch":5,"fsid":"7798f833-50d5-4df5-a39c-9dab1506b7db","created":"2026-02-22T23:02:46.276170+0000","modified":"2026-02-22T23:02:46.837523+0000","last_up_change":"0.000000","last_in_change":"2026-02-22T23:02:46.837523+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"882c47f4-e016-46e9-b67e-b2d6389fc9f8","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"9835b2d4-f84f-4f74-b135-663786caa176","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"59a03d64-5286-4376-b7d5-80007120931a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2026-02-22T23:02:47.362 DEBUG:teuthology.misc:0 of 3 OSDs are up 2026-02-22T23:02:47.375 INFO:tasks.ceph.mgr.x.trial168.stderr:2026-02-22T23:02:47.373+0000 7f07a1b62140 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2026-02-22T23:02:47.401 INFO:tasks.ceph.mgr.x.trial168.stderr:2026-02-22T23:02:47.399+0000 7f07a1b62140 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2026-02-22T23:02:47.424 INFO:tasks.ceph.mgr.x.trial168.stderr:2026-02-22T23:02:47.422+0000 7f07a1b62140 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2026-02-22T23:02:47.622 INFO:tasks.ceph.mgr.x.trial168.stderr:2026-02-22T23:02:47.620+0000 7f07a1b62140 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2026-02-22T23:02:47.677 INFO:tasks.ceph.mgr.x.trial168.stderr:2026-02-22T23:02:47.675+0000 7f07a1b62140 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2026-02-22T23:02:47.699 INFO:tasks.ceph.mgr.x.trial168.stderr:2026-02-22T23:02:47.697+0000 7f07a1b62140 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2026-02-22T23:02:47.723 INFO:tasks.ceph.mgr.x.trial168.stderr:2026-02-22T23:02:47.721+0000 7f07a1b62140 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2026-02-22T23:02:47.778 INFO:tasks.ceph.mgr.x.trial168.stderr:2026-02-22T23:02:47.775+0000 7f07a1b62140 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2026-02-22T23:02:47.799 INFO:tasks.ceph.mgr.x.trial168.stderr:2026-02-22T23:02:47.796+0000 7f07a1b62140 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2026-02-22T23:02:47.851 INFO:tasks.ceph.mgr.x.trial168.stderr:2026-02-22T23:02:47.849+0000 7f07a1b62140 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2026-02-22T23:02:48.260 INFO:tasks.ceph.mgr.x.trial168.stderr:2026-02-22T23:02:48.258+0000 7f07a1b62140 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2026-02-22T23:02:48.278 INFO:tasks.ceph.osd.2.trial168.stderr:2026-02-22T23:02:48.276+0000 7feaad39c780 -1 Falling back to public interface 2026-02-22T23:02:48.283 INFO:tasks.ceph.mgr.x.trial168.stderr:2026-02-22T23:02:48.281+0000 7f07a1b62140 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2026-02-22T23:02:48.290 INFO:tasks.ceph.osd.0.trial168.stderr:2026-02-22T23:02:48.288+0000 7fa2b4c14780 -1 Falling back to public interface 2026-02-22T23:02:48.543 INFO:tasks.ceph.osd.1.trial168.stderr:2026-02-22T23:02:48.540+0000 7f0ee3751780 -1 Falling back to public interface 2026-02-22T23:02:48.627 INFO:tasks.ceph.mgr.x.trial168.stderr:2026-02-22T23:02:48.625+0000 7f07a1b62140 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2026-02-22T23:02:49.631 INFO:tasks.ceph.mgr.x.trial168.stderr:2026-02-22T23:02:49.629+0000 7f06bf298640 -1 mgr.server handle_report got status from non-daemon mon.a 2026-02-22T23:02:52.056 INFO:tasks.ceph.osd.0.trial168.stderr:2026-02-22T23:02:52.053+0000 7fa2b4c14780 -1 osd.0 0 log_to_monitors true 2026-02-22T23:02:52.283 INFO:tasks.ceph.osd.1.trial168.stderr:2026-02-22T23:02:52.281+0000 7f0ee3751780 -1 osd.1 0 log_to_monitors true 2026-02-22T23:02:52.291 INFO:tasks.ceph.osd.2.trial168.stderr:2026-02-22T23:02:52.289+0000 7feaad39c780 -1 osd.2 0 log_to_monitors true 2026-02-22T23:02:53.651 INFO:tasks.ceph.osd.0.trial168.stderr:2026-02-22T23:02:53.649+0000 7fa2b04d4640 -1 osd.0 0 waiting for initial osdmap 2026-02-22T23:02:53.651 INFO:tasks.ceph.osd.1.trial168.stderr:2026-02-22T23:02:53.649+0000 7f0edf6de640 -1 osd.1 0 waiting for initial osdmap 2026-02-22T23:02:53.652 INFO:tasks.ceph.osd.0.trial168.stderr:2026-02-22T23:02:53.651+0000 7fa2a82f6640 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2026-02-22T23:02:53.652 INFO:tasks.ceph.osd.1.trial168.stderr:2026-02-22T23:02:53.651+0000 7f0ed7500640 -1 osd.1 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2026-02-22T23:02:53.654 INFO:tasks.ceph.osd.2.trial168.stderr:2026-02-22T23:02:53.653+0000 7feaa9329640 -1 osd.2 0 waiting for initial osdmap 2026-02-22T23:02:53.656 INFO:tasks.ceph.osd.2.trial168.stderr:2026-02-22T23:02:53.655+0000 7feaa114b640 -1 osd.2 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2026-02-22T23:02:53.665 DEBUG:teuthology.orchestra.run.trial168:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2026-02-22T23:02:53.816 INFO:teuthology.misc.health.trial168.stdout: 2026-02-22T23:02:53.816 INFO:teuthology.misc.health.trial168.stdout:{"epoch":7,"fsid":"7798f833-50d5-4df5-a39c-9dab1506b7db","created":"2026-02-22T23:02:46.276170+0000","modified":"2026-02-22T23:02:53.634519+0000","last_up_change":"0.000000","last_in_change":"2026-02-22T23:02:46.837523+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"882c47f4-e016-46e9-b67e-b2d6389fc9f8","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"9835b2d4-f84f-4f74-b135-663786caa176","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"59a03d64-5286-4376-b7d5-80007120931a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2026-02-22T23:02:53.822 DEBUG:teuthology.misc:0 of 3 OSDs are up 2026-02-22T23:03:00.125 DEBUG:teuthology.orchestra.run.trial168:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2026-02-22T23:03:00.279 INFO:teuthology.misc.health.trial168.stdout: 2026-02-22T23:03:00.279 INFO:teuthology.misc.health.trial168.stdout:{"epoch":11,"fsid":"7798f833-50d5-4df5-a39c-9dab1506b7db","created":"2026-02-22T23:02:46.276170+0000","modified":"2026-02-22T23:02:57.642536+0000","last_up_change":"2026-02-22T23:02:54.636216+0000","last_in_change":"2026-02-22T23:02:46.837523+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2026-02-22T23:02:54.645801+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"11","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.9900000095367432,"score_stable":2.9900000095367432,"optimal_score":0.67000001668930054,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"882c47f4-e016-46e9-b67e-b2d6389fc9f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.168:6808","nonce":3554969634},{"type":"v1","addr":"10.20.193.168:6809","nonce":3554969634}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.168:6810","nonce":3554969634},{"type":"v1","addr":"10.20.193.168:6811","nonce":3554969634}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.168:6814","nonce":3554969634},{"type":"v1","addr":"10.20.193.168:6815","nonce":3554969634}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.168:6812","nonce":3554969634},{"type":"v1","addr":"10.20.193.168:6813","nonce":3554969634}]},"public_addr":"10.20.193.168:6809/3554969634","cluster_addr":"10.20.193.168:6811/3554969634","heartbeat_back_addr":"10.20.193.168:6815/3554969634","heartbeat_front_addr":"10.20.193.168:6813/3554969634","state":["exists","up"]},{"osd":1,"uuid":"9835b2d4-f84f-4f74-b135-663786caa176","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":9,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.168:6816","nonce":1754802247},{"type":"v1","addr":"10.20.193.168:6817","nonce":1754802247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.168:6818","nonce":1754802247},{"type":"v1","addr":"10.20.193.168:6819","nonce":1754802247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.168:6822","nonce":1754802247},{"type":"v1","addr":"10.20.193.168:6823","nonce":1754802247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.168:6820","nonce":1754802247},{"type":"v1","addr":"10.20.193.168:6821","nonce":1754802247}]},"public_addr":"10.20.193.168:6817/1754802247","cluster_addr":"10.20.193.168:6819/1754802247","heartbeat_back_addr":"10.20.193.168:6823/1754802247","heartbeat_front_addr":"10.20.193.168:6821/1754802247","state":["exists","up"]},{"osd":2,"uuid":"59a03d64-5286-4376-b7d5-80007120931a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.168:6800","nonce":1816324481},{"type":"v1","addr":"10.20.193.168:6801","nonce":1816324481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.168:6802","nonce":1816324481},{"type":"v1","addr":"10.20.193.168:6803","nonce":1816324481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.168:6806","nonce":1816324481},{"type":"v1","addr":"10.20.193.168:6807","nonce":1816324481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.168:6804","nonce":1816324481},{"type":"v1","addr":"10.20.193.168:6805","nonce":1816324481}]},"public_addr":"10.20.193.168:6801/1816324481","cluster_addr":"10.20.193.168:6803/1816324481","heartbeat_back_addr":"10.20.193.168:6807/1816324481","heartbeat_front_addr":"10.20.193.168:6805/1816324481","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2026-02-22T23:02:53.012632+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2026-02-22T23:02:53.333411+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2026-02-22T23:02:53.320621+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2026-02-22T23:03:00.286 DEBUG:teuthology.misc:3 of 3 OSDs are up 2026-02-22T23:03:00.286 INFO:tasks.ceph:Creating RBD pool 2026-02-22T23:03:00.286 DEBUG:teuthology.orchestra.run.trial168:> sudo ceph --cluster ceph osd pool create rbd 8 2026-02-22T23:03:00.649 INFO:teuthology.orchestra.run.trial168.stderr:pool 'rbd' created 2026-02-22T23:03:00.659 DEBUG:teuthology.orchestra.run.trial168:> rbd --cluster ceph pool init rbd 2026-02-22T23:03:03.662 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2026-02-22T23:03:03.662 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2026-02-22T23:03:03.663 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2026-02-22T23:03:03.663 DEBUG:teuthology.orchestra.run.trial168:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2026-02-22T23:03:03.663 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2026-02-22T23:03:03.841 INFO:teuthology.orchestra.run.trial168.stdout: 2026-02-22T23:03:03.849 INFO:teuthology.orchestra.run.trial168.stdout:{"epoch":5,"flags":0,"active_gid":4102,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.168:6824","nonce":3199347983},{"type":"v1","addr":"10.20.193.168:6825","nonce":3199347983}]},"active_addr":"10.20.193.168:6825/3199347983","active_change":"2026-02-22T23:02:48.627448+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_pg_upmap_activity":{"name":"update_pg_upmap_activity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Updates pg_upmap activity stats to be used in `balancer status detail`","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:10.4.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.7.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.51.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"quay.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_count_max":{"name":"ssh_keepalive_count_max","type":"int","level":"advanced","flags":0,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"How many times ssh connections can fail liveness checks before the host is marked offline","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_interval":{"name":"ssh_keepalive_interval","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"How often ssh connections are checked for liveness","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":true,"error_string":"","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_requests":{"name":"max_requests","type":"int","level":"advanced","flags":0,"default_value":"500","min":"","max":"","enum_allowed":[],"desc":"Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number. if un-finished request is removed, error message will be logged in the ceph-mgr log.","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"force_disabled_modules":{},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"10.20.193.168:0","nonce":407439883}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"10.20.193.168:0","nonce":1833326642}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"10.20.193.168:0","nonce":1120003767}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"10.20.193.168:0","nonce":3032075462}]}]} 2026-02-22T23:03:03.851 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2026-02-22T23:03:03.851 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2026-02-22T23:03:03.851 DEBUG:teuthology.orchestra.run.trial168:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2026-02-22T23:03:04.008 INFO:teuthology.orchestra.run.trial168.stdout: 2026-02-22T23:03:04.008 INFO:teuthology.orchestra.run.trial168.stdout:{"epoch":15,"fsid":"7798f833-50d5-4df5-a39c-9dab1506b7db","created":"2026-02-22T23:02:46.276170+0000","modified":"2026-02-22T23:03:03.653056+0000","last_up_change":"2026-02-22T23:02:54.636216+0000","last_in_change":"2026-02-22T23:02:46.837523+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2026-02-22T23:02:54.645801+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"11","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.9900000095367432,"score_stable":2.9900000095367432,"optimal_score":0.67000001668930054,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2026-02-22T23:03:00.444070+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":15,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"882c47f4-e016-46e9-b67e-b2d6389fc9f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":12,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.168:6808","nonce":3554969634},{"type":"v1","addr":"10.20.193.168:6809","nonce":3554969634}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.168:6810","nonce":3554969634},{"type":"v1","addr":"10.20.193.168:6811","nonce":3554969634}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.168:6814","nonce":3554969634},{"type":"v1","addr":"10.20.193.168:6815","nonce":3554969634}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.168:6812","nonce":3554969634},{"type":"v1","addr":"10.20.193.168:6813","nonce":3554969634}]},"public_addr":"10.20.193.168:6809/3554969634","cluster_addr":"10.20.193.168:6811/3554969634","heartbeat_back_addr":"10.20.193.168:6815/3554969634","heartbeat_front_addr":"10.20.193.168:6813/3554969634","state":["exists","up"]},{"osd":1,"uuid":"9835b2d4-f84f-4f74-b135-663786caa176","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":12,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.168:6816","nonce":1754802247},{"type":"v1","addr":"10.20.193.168:6817","nonce":1754802247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.168:6818","nonce":1754802247},{"type":"v1","addr":"10.20.193.168:6819","nonce":1754802247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.168:6822","nonce":1754802247},{"type":"v1","addr":"10.20.193.168:6823","nonce":1754802247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.168:6820","nonce":1754802247},{"type":"v1","addr":"10.20.193.168:6821","nonce":1754802247}]},"public_addr":"10.20.193.168:6817/1754802247","cluster_addr":"10.20.193.168:6819/1754802247","heartbeat_back_addr":"10.20.193.168:6823/1754802247","heartbeat_front_addr":"10.20.193.168:6821/1754802247","state":["exists","up"]},{"osd":2,"uuid":"59a03d64-5286-4376-b7d5-80007120931a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":12,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.168:6800","nonce":1816324481},{"type":"v1","addr":"10.20.193.168:6801","nonce":1816324481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.168:6802","nonce":1816324481},{"type":"v1","addr":"10.20.193.168:6803","nonce":1816324481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.168:6806","nonce":1816324481},{"type":"v1","addr":"10.20.193.168:6807","nonce":1816324481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.168:6804","nonce":1816324481},{"type":"v1","addr":"10.20.193.168:6805","nonce":1816324481}]},"public_addr":"10.20.193.168:6801/1816324481","cluster_addr":"10.20.193.168:6803/1816324481","heartbeat_back_addr":"10.20.193.168:6807/1816324481","heartbeat_front_addr":"10.20.193.168:6805/1816324481","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2026-02-22T23:02:53.012632+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2026-02-22T23:02:53.333411+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2026-02-22T23:02:53.320621+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_removed_snaps":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2026-02-22T23:03:04.017 INFO:tasks.ceph.ceph_manager.ceph:all up! 2026-02-22T23:03:04.017 DEBUG:teuthology.orchestra.run.trial168:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2026-02-22T23:03:04.175 INFO:teuthology.orchestra.run.trial168.stdout: 2026-02-22T23:03:04.175 INFO:teuthology.orchestra.run.trial168.stdout:{"epoch":15,"fsid":"7798f833-50d5-4df5-a39c-9dab1506b7db","created":"2026-02-22T23:02:46.276170+0000","modified":"2026-02-22T23:03:03.653056+0000","last_up_change":"2026-02-22T23:02:54.636216+0000","last_in_change":"2026-02-22T23:02:46.837523+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2026-02-22T23:02:54.645801+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"11","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.9900000095367432,"score_stable":2.9900000095367432,"optimal_score":0.67000001668930054,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2026-02-22T23:03:00.444070+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":15,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"882c47f4-e016-46e9-b67e-b2d6389fc9f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":12,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.168:6808","nonce":3554969634},{"type":"v1","addr":"10.20.193.168:6809","nonce":3554969634}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.168:6810","nonce":3554969634},{"type":"v1","addr":"10.20.193.168:6811","nonce":3554969634}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.168:6814","nonce":3554969634},{"type":"v1","addr":"10.20.193.168:6815","nonce":3554969634}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.168:6812","nonce":3554969634},{"type":"v1","addr":"10.20.193.168:6813","nonce":3554969634}]},"public_addr":"10.20.193.168:6809/3554969634","cluster_addr":"10.20.193.168:6811/3554969634","heartbeat_back_addr":"10.20.193.168:6815/3554969634","heartbeat_front_addr":"10.20.193.168:6813/3554969634","state":["exists","up"]},{"osd":1,"uuid":"9835b2d4-f84f-4f74-b135-663786caa176","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":12,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.168:6816","nonce":1754802247},{"type":"v1","addr":"10.20.193.168:6817","nonce":1754802247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.168:6818","nonce":1754802247},{"type":"v1","addr":"10.20.193.168:6819","nonce":1754802247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.168:6822","nonce":1754802247},{"type":"v1","addr":"10.20.193.168:6823","nonce":1754802247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.168:6820","nonce":1754802247},{"type":"v1","addr":"10.20.193.168:6821","nonce":1754802247}]},"public_addr":"10.20.193.168:6817/1754802247","cluster_addr":"10.20.193.168:6819/1754802247","heartbeat_back_addr":"10.20.193.168:6823/1754802247","heartbeat_front_addr":"10.20.193.168:6821/1754802247","state":["exists","up"]},{"osd":2,"uuid":"59a03d64-5286-4376-b7d5-80007120931a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":12,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.168:6800","nonce":1816324481},{"type":"v1","addr":"10.20.193.168:6801","nonce":1816324481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.168:6802","nonce":1816324481},{"type":"v1","addr":"10.20.193.168:6803","nonce":1816324481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.168:6806","nonce":1816324481},{"type":"v1","addr":"10.20.193.168:6807","nonce":1816324481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.168:6804","nonce":1816324481},{"type":"v1","addr":"10.20.193.168:6805","nonce":1816324481}]},"public_addr":"10.20.193.168:6801/1816324481","cluster_addr":"10.20.193.168:6803/1816324481","heartbeat_back_addr":"10.20.193.168:6807/1816324481","heartbeat_front_addr":"10.20.193.168:6805/1816324481","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2026-02-22T23:02:53.012632+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2026-02-22T23:02:53.333411+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2026-02-22T23:02:53.320621+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_removed_snaps":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2026-02-22T23:03:04.183 DEBUG:teuthology.orchestra.run.trial168:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2026-02-22T23:03:04.183 DEBUG:teuthology.orchestra.run.trial168:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2026-02-22T23:03:04.183 DEBUG:teuthology.orchestra.run.trial168:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2026-02-22T23:03:04.268 INFO:teuthology.orchestra.run.trial168.stdout:34359738372 2026-02-22T23:03:04.269 DEBUG:teuthology.orchestra.run.trial168:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2026-02-22T23:03:04.269 INFO:teuthology.orchestra.run.trial168.stdout:34359738373 2026-02-22T23:03:04.269 DEBUG:teuthology.orchestra.run.trial168:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2026-02-22T23:03:04.270 INFO:teuthology.orchestra.run.trial168.stdout:34359738372 2026-02-22T23:03:04.270 DEBUG:teuthology.orchestra.run.trial168:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2026-02-22T23:03:04.430 INFO:teuthology.orchestra.run.trial168.stdout:34359738371 2026-02-22T23:03:04.431 INFO:teuthology.orchestra.run.trial168.stdout:34359738372 2026-02-22T23:03:04.435 INFO:teuthology.orchestra.run.trial168.stdout:34359738371 2026-02-22T23:03:04.440 INFO:tasks.ceph.ceph_manager.ceph:need seq 34359738372 got 34359738371 for osd.0 2026-02-22T23:03:04.441 INFO:tasks.ceph.ceph_manager.ceph:need seq 34359738373 got 34359738372 for osd.2 2026-02-22T23:03:04.445 INFO:tasks.ceph.ceph_manager.ceph:need seq 34359738372 got 34359738371 for osd.1 2026-02-22T23:03:05.440 DEBUG:teuthology.orchestra.run.trial168:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2026-02-22T23:03:05.441 DEBUG:teuthology.orchestra.run.trial168:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2026-02-22T23:03:05.445 DEBUG:teuthology.orchestra.run.trial168:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2026-02-22T23:03:05.601 INFO:teuthology.orchestra.run.trial168.stdout:34359738372 2026-02-22T23:03:05.606 INFO:teuthology.orchestra.run.trial168.stdout:34359738373 2026-02-22T23:03:05.610 INFO:tasks.ceph.ceph_manager.ceph:need seq 34359738372 got 34359738372 for osd.0 2026-02-22T23:03:05.610 DEBUG:teuthology.parallel:result is None 2026-02-22T23:03:05.616 INFO:tasks.ceph.ceph_manager.ceph:need seq 34359738373 got 34359738373 for osd.2 2026-02-22T23:03:05.616 DEBUG:teuthology.parallel:result is None 2026-02-22T23:03:05.645 INFO:teuthology.orchestra.run.trial168.stdout:34359738372 2026-02-22T23:03:05.654 INFO:tasks.ceph.ceph_manager.ceph:need seq 34359738372 got 34359738372 for osd.1 2026-02-22T23:03:05.654 DEBUG:teuthology.parallel:result is None 2026-02-22T23:03:05.654 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2026-02-22T23:03:05.655 DEBUG:teuthology.orchestra.run.trial168:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T23:03:05.850 INFO:teuthology.orchestra.run.trial168.stdout: 2026-02-22T23:03:05.851 INFO:teuthology.orchestra.run.trial168.stderr:dumped all 2026-02-22T23:03:05.858 INFO:teuthology.orchestra.run.trial168.stdout:{"pg_ready":true,"pg_map":{"version":18,"stamp":"2026-02-22T23:03:04.632279+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":103,"num_write_kb":1630,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":75,"ondisk_log_size":75,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":15,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":82440,"kb_used_data":1544,"kb_used_omap":4,"kb_used_meta":80827,"kb_avail":283033080,"statfs":{"total":289910292480,"available":289825873920,"internally_reserved":0,"allocated":1581056,"data_stored":1292339,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4782,"internal_metadata":82767186},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"3.982128"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":19,"reported_epoch":15,"state":"active+clean","last_fresh":"2026-02-22T23:03:03.657748+0000","last_change":"2026-02-22T23:03:03.657748+0000","last_active":"2026-02-22T23:03:03.657748+0000","last_peered":"2026-02-22T23:03:03.657748+0000","last_clean":"2026-02-22T23:03:03.657748+0000","last_became_active":"2026-02-22T23:03:01.655313+0000","last_became_peered":"2026-02-22T23:03:01.655313+0000","last_unstale":"2026-02-22T23:03:03.657748+0000","last_undegraded":"2026-02-22T23:03:03.657748+0000","last_fullsized":"2026-02-22T23:03:03.657748+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T23:03:00.647409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T23:03:00.647409+0000","last_clean_scrub_stamp":"2026-02-22T23:03:00.647409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T06:55:02.798991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077562300000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":19,"reported_epoch":15,"state":"active+clean","last_fresh":"2026-02-22T23:03:03.657941+0000","last_change":"2026-02-22T23:03:03.657941+0000","last_active":"2026-02-22T23:03:03.657941+0000","last_peered":"2026-02-22T23:03:03.657941+0000","last_clean":"2026-02-22T23:03:03.657941+0000","last_became_active":"2026-02-22T23:03:01.654932+0000","last_became_peered":"2026-02-22T23:03:01.654932+0000","last_unstale":"2026-02-22T23:03:03.657941+0000","last_undegraded":"2026-02-22T23:03:03.657941+0000","last_fullsized":"2026-02-22T23:03:03.657941+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T23:03:00.647409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T23:03:00.647409+0000","last_clean_scrub_stamp":"2026-02-22T23:03:00.647409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T00:40:58.786037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00092843799999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":19,"reported_epoch":15,"state":"active+clean","last_fresh":"2026-02-22T23:03:03.657408+0000","last_change":"2026-02-22T23:03:03.657408+0000","last_active":"2026-02-22T23:03:03.657408+0000","last_peered":"2026-02-22T23:03:03.657408+0000","last_clean":"2026-02-22T23:03:03.657408+0000","last_became_active":"2026-02-22T23:03:01.655252+0000","last_became_peered":"2026-02-22T23:03:01.655252+0000","last_unstale":"2026-02-22T23:03:03.657408+0000","last_undegraded":"2026-02-22T23:03:03.657408+0000","last_fullsized":"2026-02-22T23:03:03.657408+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T23:03:00.647409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T23:03:00.647409+0000","last_clean_scrub_stamp":"2026-02-22T23:03:00.647409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T08:32:13.395954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00046171899999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":19,"reported_epoch":15,"state":"active+clean","last_fresh":"2026-02-22T23:03:03.657389+0000","last_change":"2026-02-22T23:03:03.657389+0000","last_active":"2026-02-22T23:03:03.657389+0000","last_peered":"2026-02-22T23:03:03.657389+0000","last_clean":"2026-02-22T23:03:03.657389+0000","last_became_active":"2026-02-22T23:03:01.654989+0000","last_became_peered":"2026-02-22T23:03:01.654989+0000","last_unstale":"2026-02-22T23:03:03.657389+0000","last_undegraded":"2026-02-22T23:03:03.657389+0000","last_fullsized":"2026-02-22T23:03:03.657389+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T23:03:00.647409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T23:03:00.647409+0000","last_clean_scrub_stamp":"2026-02-22T23:03:00.647409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T01:06:44.014188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000472108,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.2","version":"15'2","reported_seq":21,"reported_epoch":15,"state":"active+clean","last_fresh":"2026-02-22T23:03:03.656553+0000","last_change":"2026-02-22T23:03:03.656168+0000","last_active":"2026-02-22T23:03:03.656553+0000","last_peered":"2026-02-22T23:03:03.656553+0000","last_clean":"2026-02-22T23:03:03.656553+0000","last_became_active":"2026-02-22T23:03:01.655020+0000","last_became_peered":"2026-02-22T23:03:01.655020+0000","last_unstale":"2026-02-22T23:03:03.656553+0000","last_undegraded":"2026-02-22T23:03:03.656553+0000","last_fullsized":"2026-02-22T23:03:03.656553+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T23:03:00.647409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T23:03:00.647409+0000","last_clean_scrub_stamp":"2026-02-22T23:03:00.647409+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T01:25:19.891118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000724969,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":19,"reported_epoch":15,"state":"active+clean","last_fresh":"2026-02-22T23:03:04.197991+0000","last_change":"2026-02-22T23:03:04.197991+0000","last_active":"2026-02-22T23:03:04.197991+0000","last_peered":"2026-02-22T23:03:04.197991+0000","last_clean":"2026-02-22T23:03:04.197991+0000","last_became_active":"2026-02-22T23:03:01.655046+0000","last_became_peered":"2026-02-22T23:03:01.655046+0000","last_unstale":"2026-02-22T23:03:04.197991+0000","last_undegraded":"2026-02-22T23:03:04.197991+0000","last_fullsized":"2026-02-22T23:03:04.197991+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T23:03:00.647409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T23:03:00.647409+0000","last_clean_scrub_stamp":"2026-02-22T23:03:00.647409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T05:07:31.304712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00038590800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":19,"reported_epoch":15,"state":"active+clean","last_fresh":"2026-02-22T23:03:04.197929+0000","last_change":"2026-02-22T23:03:04.197929+0000","last_active":"2026-02-22T23:03:04.197929+0000","last_peered":"2026-02-22T23:03:04.197929+0000","last_clean":"2026-02-22T23:03:04.197929+0000","last_became_active":"2026-02-22T23:03:01.655068+0000","last_became_peered":"2026-02-22T23:03:01.655068+0000","last_unstale":"2026-02-22T23:03:04.197929+0000","last_undegraded":"2026-02-22T23:03:04.197929+0000","last_fullsized":"2026-02-22T23:03:04.197929+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T23:03:00.647409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T23:03:00.647409+0000","last_clean_scrub_stamp":"2026-02-22T23:03:00.647409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T00:00:11.473581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00033282899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.3","version":"13'1","reported_seq":20,"reported_epoch":15,"state":"active+clean","last_fresh":"2026-02-22T23:03:03.657709+0000","last_change":"2026-02-22T23:03:03.657709+0000","last_active":"2026-02-22T23:03:03.657709+0000","last_peered":"2026-02-22T23:03:03.657709+0000","last_clean":"2026-02-22T23:03:03.657709+0000","last_became_active":"2026-02-22T23:03:01.654744+0000","last_became_peered":"2026-02-22T23:03:01.654744+0000","last_unstale":"2026-02-22T23:03:03.657709+0000","last_undegraded":"2026-02-22T23:03:03.657709+0000","last_fullsized":"2026-02-22T23:03:03.657709+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T23:03:00.647409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T23:03:00.647409+0000","last_clean_scrub_stamp":"2026-02-22T23:03:00.647409+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T08:30:18.919810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00079898700000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"10'72","reported_seq":110,"reported_epoch":15,"state":"active+clean","last_fresh":"2026-02-22T23:03:03.656713+0000","last_change":"2026-02-22T23:02:56.646381+0000","last_active":"2026-02-22T23:03:03.656713+0000","last_peered":"2026-02-22T23:03:03.656713+0000","last_clean":"2026-02-22T23:03:03.656713+0000","last_became_active":"2026-02-22T23:02:56.645752+0000","last_became_peered":"2026-02-22T23:02:56.645752+0000","last_unstale":"2026-02-22T23:03:03.656713+0000","last_undegraded":"2026-02-22T23:03:03.656713+0000","last_fullsized":"2026-02-22T23:03:03.656713+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T23:02:55.640202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T23:02:55.640202+0000","last_clean_scrub_stamp":"2026-02-22T23:02:55.640202+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T05:06:17.108040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359738373,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27088,"kb_used_data":120,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344752,"statfs":{"total":96636764160,"available":96609026048,"internally_reserved":0,"allocated":122880,"data_stored":34815,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27589062},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":8,"seq":34359738372,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27676,"kb_used_data":712,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344164,"statfs":{"total":96636764160,"available":96608423936,"internally_reserved":0,"allocated":729088,"data_stored":628762,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27589062},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738372,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27676,"kb_used_data":712,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344164,"statfs":{"total":96636764160,"available":96608423936,"internally_reserved":0,"allocated":729088,"data_stored":628762,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27589062},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T23:03:05.860 DEBUG:teuthology.orchestra.run.trial168:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T23:03:06.016 INFO:teuthology.orchestra.run.trial168.stdout: 2026-02-22T23:03:06.017 INFO:teuthology.orchestra.run.trial168.stderr:dumped all 2026-02-22T23:03:06.025 INFO:teuthology.orchestra.run.trial168.stdout:{"pg_ready":true,"pg_map":{"version":18,"stamp":"2026-02-22T23:03:04.632279+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":103,"num_write_kb":1630,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":75,"ondisk_log_size":75,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":15,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":82440,"kb_used_data":1544,"kb_used_omap":4,"kb_used_meta":80827,"kb_avail":283033080,"statfs":{"total":289910292480,"available":289825873920,"internally_reserved":0,"allocated":1581056,"data_stored":1292339,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4782,"internal_metadata":82767186},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"3.982128"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":19,"reported_epoch":15,"state":"active+clean","last_fresh":"2026-02-22T23:03:03.657748+0000","last_change":"2026-02-22T23:03:03.657748+0000","last_active":"2026-02-22T23:03:03.657748+0000","last_peered":"2026-02-22T23:03:03.657748+0000","last_clean":"2026-02-22T23:03:03.657748+0000","last_became_active":"2026-02-22T23:03:01.655313+0000","last_became_peered":"2026-02-22T23:03:01.655313+0000","last_unstale":"2026-02-22T23:03:03.657748+0000","last_undegraded":"2026-02-22T23:03:03.657748+0000","last_fullsized":"2026-02-22T23:03:03.657748+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T23:03:00.647409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T23:03:00.647409+0000","last_clean_scrub_stamp":"2026-02-22T23:03:00.647409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T06:55:02.798991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077562300000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":19,"reported_epoch":15,"state":"active+clean","last_fresh":"2026-02-22T23:03:03.657941+0000","last_change":"2026-02-22T23:03:03.657941+0000","last_active":"2026-02-22T23:03:03.657941+0000","last_peered":"2026-02-22T23:03:03.657941+0000","last_clean":"2026-02-22T23:03:03.657941+0000","last_became_active":"2026-02-22T23:03:01.654932+0000","last_became_peered":"2026-02-22T23:03:01.654932+0000","last_unstale":"2026-02-22T23:03:03.657941+0000","last_undegraded":"2026-02-22T23:03:03.657941+0000","last_fullsized":"2026-02-22T23:03:03.657941+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T23:03:00.647409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T23:03:00.647409+0000","last_clean_scrub_stamp":"2026-02-22T23:03:00.647409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T00:40:58.786037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00092843799999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":19,"reported_epoch":15,"state":"active+clean","last_fresh":"2026-02-22T23:03:03.657408+0000","last_change":"2026-02-22T23:03:03.657408+0000","last_active":"2026-02-22T23:03:03.657408+0000","last_peered":"2026-02-22T23:03:03.657408+0000","last_clean":"2026-02-22T23:03:03.657408+0000","last_became_active":"2026-02-22T23:03:01.655252+0000","last_became_peered":"2026-02-22T23:03:01.655252+0000","last_unstale":"2026-02-22T23:03:03.657408+0000","last_undegraded":"2026-02-22T23:03:03.657408+0000","last_fullsized":"2026-02-22T23:03:03.657408+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T23:03:00.647409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T23:03:00.647409+0000","last_clean_scrub_stamp":"2026-02-22T23:03:00.647409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T08:32:13.395954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00046171899999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":19,"reported_epoch":15,"state":"active+clean","last_fresh":"2026-02-22T23:03:03.657389+0000","last_change":"2026-02-22T23:03:03.657389+0000","last_active":"2026-02-22T23:03:03.657389+0000","last_peered":"2026-02-22T23:03:03.657389+0000","last_clean":"2026-02-22T23:03:03.657389+0000","last_became_active":"2026-02-22T23:03:01.654989+0000","last_became_peered":"2026-02-22T23:03:01.654989+0000","last_unstale":"2026-02-22T23:03:03.657389+0000","last_undegraded":"2026-02-22T23:03:03.657389+0000","last_fullsized":"2026-02-22T23:03:03.657389+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T23:03:00.647409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T23:03:00.647409+0000","last_clean_scrub_stamp":"2026-02-22T23:03:00.647409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T01:06:44.014188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000472108,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.2","version":"15'2","reported_seq":21,"reported_epoch":15,"state":"active+clean","last_fresh":"2026-02-22T23:03:03.656553+0000","last_change":"2026-02-22T23:03:03.656168+0000","last_active":"2026-02-22T23:03:03.656553+0000","last_peered":"2026-02-22T23:03:03.656553+0000","last_clean":"2026-02-22T23:03:03.656553+0000","last_became_active":"2026-02-22T23:03:01.655020+0000","last_became_peered":"2026-02-22T23:03:01.655020+0000","last_unstale":"2026-02-22T23:03:03.656553+0000","last_undegraded":"2026-02-22T23:03:03.656553+0000","last_fullsized":"2026-02-22T23:03:03.656553+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T23:03:00.647409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T23:03:00.647409+0000","last_clean_scrub_stamp":"2026-02-22T23:03:00.647409+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T01:25:19.891118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000724969,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":19,"reported_epoch":15,"state":"active+clean","last_fresh":"2026-02-22T23:03:04.197991+0000","last_change":"2026-02-22T23:03:04.197991+0000","last_active":"2026-02-22T23:03:04.197991+0000","last_peered":"2026-02-22T23:03:04.197991+0000","last_clean":"2026-02-22T23:03:04.197991+0000","last_became_active":"2026-02-22T23:03:01.655046+0000","last_became_peered":"2026-02-22T23:03:01.655046+0000","last_unstale":"2026-02-22T23:03:04.197991+0000","last_undegraded":"2026-02-22T23:03:04.197991+0000","last_fullsized":"2026-02-22T23:03:04.197991+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T23:03:00.647409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T23:03:00.647409+0000","last_clean_scrub_stamp":"2026-02-22T23:03:00.647409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T05:07:31.304712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00038590800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":19,"reported_epoch":15,"state":"active+clean","last_fresh":"2026-02-22T23:03:04.197929+0000","last_change":"2026-02-22T23:03:04.197929+0000","last_active":"2026-02-22T23:03:04.197929+0000","last_peered":"2026-02-22T23:03:04.197929+0000","last_clean":"2026-02-22T23:03:04.197929+0000","last_became_active":"2026-02-22T23:03:01.655068+0000","last_became_peered":"2026-02-22T23:03:01.655068+0000","last_unstale":"2026-02-22T23:03:04.197929+0000","last_undegraded":"2026-02-22T23:03:04.197929+0000","last_fullsized":"2026-02-22T23:03:04.197929+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T23:03:00.647409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T23:03:00.647409+0000","last_clean_scrub_stamp":"2026-02-22T23:03:00.647409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T00:00:11.473581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00033282899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.3","version":"13'1","reported_seq":20,"reported_epoch":15,"state":"active+clean","last_fresh":"2026-02-22T23:03:03.657709+0000","last_change":"2026-02-22T23:03:03.657709+0000","last_active":"2026-02-22T23:03:03.657709+0000","last_peered":"2026-02-22T23:03:03.657709+0000","last_clean":"2026-02-22T23:03:03.657709+0000","last_became_active":"2026-02-22T23:03:01.654744+0000","last_became_peered":"2026-02-22T23:03:01.654744+0000","last_unstale":"2026-02-22T23:03:03.657709+0000","last_undegraded":"2026-02-22T23:03:03.657709+0000","last_fullsized":"2026-02-22T23:03:03.657709+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T23:03:00.647409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T23:03:00.647409+0000","last_clean_scrub_stamp":"2026-02-22T23:03:00.647409+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T08:30:18.919810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00079898700000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"10'72","reported_seq":110,"reported_epoch":15,"state":"active+clean","last_fresh":"2026-02-22T23:03:03.656713+0000","last_change":"2026-02-22T23:02:56.646381+0000","last_active":"2026-02-22T23:03:03.656713+0000","last_peered":"2026-02-22T23:03:03.656713+0000","last_clean":"2026-02-22T23:03:03.656713+0000","last_became_active":"2026-02-22T23:02:56.645752+0000","last_became_peered":"2026-02-22T23:02:56.645752+0000","last_unstale":"2026-02-22T23:03:03.656713+0000","last_undegraded":"2026-02-22T23:03:03.656713+0000","last_fullsized":"2026-02-22T23:03:03.656713+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T23:02:55.640202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T23:02:55.640202+0000","last_clean_scrub_stamp":"2026-02-22T23:02:55.640202+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T05:06:17.108040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359738373,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27088,"kb_used_data":120,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344752,"statfs":{"total":96636764160,"available":96609026048,"internally_reserved":0,"allocated":122880,"data_stored":34815,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27589062},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":8,"seq":34359738372,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27676,"kb_used_data":712,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344164,"statfs":{"total":96636764160,"available":96608423936,"internally_reserved":0,"allocated":729088,"data_stored":628762,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27589062},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738372,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27676,"kb_used_data":712,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94344164,"statfs":{"total":96636764160,"available":96608423936,"internally_reserved":0,"allocated":729088,"data_stored":628762,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27589062},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T23:03:06.027 INFO:tasks.ceph.ceph_manager.ceph:clean! 2026-02-22T23:03:06.027 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2026-02-22T23:03:06.028 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2026-02-22T23:03:06.028 DEBUG:teuthology.orchestra.run.trial168:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-22T23:03:06.200 INFO:teuthology.orchestra.run.trial168.stdout: 2026-02-22T23:03:06.200 INFO:teuthology.orchestra.run.trial168.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2026-02-22T23:03:06.208 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2026-02-22T23:03:06.208 INFO:teuthology.run_tasks:Running task workunit... 2026-02-22T23:03:06.234 INFO:tasks.workunit:Pulling workunits from ref f90f4eaebb28b619cf76e80cbbe997656da69579 2026-02-22T23:03:06.235 INFO:tasks.workunit:Making a separate scratch dir for every client... 2026-02-22T23:03:06.235 DEBUG:teuthology.orchestra.run.trial168:> stat -- /home/ubuntu/cephtest/mnt.0 2026-02-22T23:03:06.252 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-22T23:03:06.252 INFO:teuthology.orchestra.run.trial168.stderr:stat: cannot statx '/home/ubuntu/cephtest/mnt.0': No such file or directory 2026-02-22T23:03:06.253 DEBUG:teuthology.orchestra.run.trial168:> mkdir -- /home/ubuntu/cephtest/mnt.0 2026-02-22T23:03:06.309 INFO:tasks.workunit:Created dir /home/ubuntu/cephtest/mnt.0 2026-02-22T23:03:06.309 DEBUG:teuthology.orchestra.run.trial168:> cd -- /home/ubuntu/cephtest/mnt.0 && mkdir -- client.0 2026-02-22T23:03:06.365 INFO:tasks.workunit:timeout=3h 2026-02-22T23:03:06.366 INFO:tasks.workunit:cleanup=True 2026-02-22T23:03:06.366 DEBUG:teuthology.orchestra.run.trial168:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://github.com/ceph/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout f90f4eaebb28b619cf76e80cbbe997656da69579 2026-02-22T23:03:06.429 INFO:tasks.workunit.client.0.trial168.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2026-02-22T23:03:27.054 INFO:tasks.workunit.client.0.trial168.stderr:Note: switching to 'f90f4eaebb28b619cf76e80cbbe997656da69579'. 2026-02-22T23:03:27.054 INFO:tasks.workunit.client.0.trial168.stderr: 2026-02-22T23:03:27.054 INFO:tasks.workunit.client.0.trial168.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2026-02-22T23:03:27.054 INFO:tasks.workunit.client.0.trial168.stderr:changes and commit them, and you can discard any commits you make in this 2026-02-22T23:03:27.054 INFO:tasks.workunit.client.0.trial168.stderr:state without impacting any branches by switching back to a branch. 2026-02-22T23:03:27.055 INFO:tasks.workunit.client.0.trial168.stderr: 2026-02-22T23:03:27.055 INFO:tasks.workunit.client.0.trial168.stderr:If you want to create a new branch to retain commits you create, you may 2026-02-22T23:03:27.055 INFO:tasks.workunit.client.0.trial168.stderr:do so (now or later) by using -c with the switch command. Example: 2026-02-22T23:03:27.055 INFO:tasks.workunit.client.0.trial168.stderr: 2026-02-22T23:03:27.055 INFO:tasks.workunit.client.0.trial168.stderr: git switch -c 2026-02-22T23:03:27.055 INFO:tasks.workunit.client.0.trial168.stderr: 2026-02-22T23:03:27.055 INFO:tasks.workunit.client.0.trial168.stderr:Or undo this operation with: 2026-02-22T23:03:27.055 INFO:tasks.workunit.client.0.trial168.stderr: 2026-02-22T23:03:27.055 INFO:tasks.workunit.client.0.trial168.stderr: git switch - 2026-02-22T23:03:27.055 INFO:tasks.workunit.client.0.trial168.stderr: 2026-02-22T23:03:27.055 INFO:tasks.workunit.client.0.trial168.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2026-02-22T23:03:27.055 INFO:tasks.workunit.client.0.trial168.stderr: 2026-02-22T23:03:27.056 INFO:tasks.workunit.client.0.trial168.stderr:HEAD is now at f90f4eaebb2 Merge branch 'wip-74712-squid' of https://github.com/idryomov/ceph into wip-yuri6-testing-2026-02-22-1629-squid 2026-02-22T23:03:27.061 DEBUG:teuthology.orchestra.run.trial168:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2026-02-22T23:03:27.119 INFO:tasks.workunit.client.0.trial168.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2026-02-22T23:03:27.120 INFO:tasks.workunit.client.0.trial168.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2026-02-22T23:03:27.120 INFO:tasks.workunit.client.0.trial168.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2026-02-22T23:03:27.212 INFO:tasks.workunit.client.0.trial168.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2026-02-22T23:03:27.242 INFO:tasks.workunit.client.0.trial168.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2026-02-22T23:03:27.269 INFO:tasks.workunit.client.0.trial168.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2026-02-22T23:03:27.270 INFO:tasks.workunit.client.0.trial168.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2026-02-22T23:03:27.271 INFO:tasks.workunit.client.0.trial168.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2026-02-22T23:03:27.296 INFO:tasks.workunit.client.0.trial168.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2026-02-22T23:03:27.300 DEBUG:teuthology.orchestra.run.trial168:> set -ex 2026-02-22T23:03:27.300 DEBUG:teuthology.orchestra.run.trial168:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2026-02-22T23:03:27.358 INFO:tasks.workunit:Running workunits matching rbd/test_librbd_python.sh on client.0... 2026-02-22T23:03:27.361 INFO:tasks.workunit:Running workunit rbd/test_librbd_python.sh... 2026-02-22T23:03:27.361 DEBUG:teuthology.orchestra.run.trial168:workunit test rbd/test_librbd_python.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=f90f4eaebb28b619cf76e80cbbe997656da69579 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 RBD_FEATURES=1 VALGRIND='--tool=memcheck --leak-check=full' adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rbd/test_librbd_python.sh 2026-02-22T23:03:27.419 INFO:tasks.workunit.client.0.trial168.stderr:++ dirname /home/ubuntu/cephtest/clone.client.0/qa/workunits/rbd/test_librbd_python.sh 2026-02-22T23:03:27.420 INFO:tasks.workunit.client.0.trial168.stderr:+ relpath=/home/ubuntu/cephtest/clone.client.0/qa/workunits/rbd/../../../src/test/pybind 2026-02-22T23:03:27.420 INFO:tasks.workunit.client.0.trial168.stderr:+ '[' -n '--tool=memcheck --leak-check=full' ']' 2026-02-22T23:03:27.421 INFO:tasks.workunit.client.0.trial168.stderr:+ valgrind --tool=memcheck --leak-check=full --suppressions=/home/ubuntu/cephtest/valgrind.supp --errors-for-leak-kinds=definite --error-exitcode=1 python3 -m pytest -v /home/ubuntu/cephtest/clone.client.0/qa/workunits/rbd/../../../src/test/pybind/test_rbd.py 2026-02-22T23:03:27.430 INFO:tasks.workunit.client.0.trial168.stderr:==40121== Memcheck, a memory error detector 2026-02-22T23:03:27.430 INFO:tasks.workunit.client.0.trial168.stderr:==40121== Copyright (C) 2002-2024, and GNU GPL'd, by Julian Seward et al. 2026-02-22T23:03:27.430 INFO:tasks.workunit.client.0.trial168.stderr:==40121== Using Valgrind-3.26.0 and LibVEX; rerun with -h for copyright info 2026-02-22T23:03:27.430 INFO:tasks.workunit.client.0.trial168.stderr:==40121== Command: python3 -m pytest -v /home/ubuntu/cephtest/clone.client.0/qa/workunits/rbd/../../../src/test/pybind/test_rbd.py 2026-02-22T23:03:27.430 INFO:tasks.workunit.client.0.trial168.stderr:==40121== 2026-02-22T23:03:38.978 INFO:tasks.workunit.client.0.trial168.stdout:============================= test session starts ============================== 2026-02-22T23:03:38.978 INFO:tasks.workunit.client.0.trial168.stdout:platform linux -- Python 3.9.25, pytest-6.2.2, py-1.10.0, pluggy-0.13.1 -- /usr/bin/python3 2026-02-22T23:03:38.978 INFO:tasks.workunit.client.0.trial168.stdout:cachedir: .pytest_cache 2026-02-22T23:03:38.978 INFO:tasks.workunit.client.0.trial168.stdout:rootdir: /home/ubuntu/cephtest/clone.client.0/src/test/pybind, configfile: pytest.ini 2026-02-22T23:04:17.862 INFO:tasks.workunit.client.0.trial168.stdout:collecting ... collected 192 items 2026-02-22T23:04:17.863 INFO:tasks.workunit.client.0.trial168.stdout: 2026-02-22T23:04:22.358 INFO:tasks.workunit.client.0.trial168.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::test_version PASSED [ 0%] 2026-02-22T23:04:22.853 INFO:tasks.workunit.client.0.trial168.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::test_create PASSED [ 1%] 2026-02-22T23:04:25.197 INFO:tasks.workunit.client.0.trial168.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::test_create_defaults PASSED [ 1%] 2026-02-22T23:04:26.012 INFO:tasks.workunit.client.0.trial168.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::test_context_manager PASSED [ 2%] 2026-02-22T23:04:28.549 INFO:tasks.workunit.client.0.trial168.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::test_open_read_only PASSED [ 2%] 2026-02-22T23:04:32.689 INFO:tasks.workunit.client.0.trial168.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::test_open_dne PASSED [ 3%] 2026-02-22T23:04:36.765 INFO:tasks.workunit.client.0.trial168.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::test_open_readonly_dne PASSED [ 3%] 2026-02-22T23:04:37.382 INFO:tasks.workunit.client.0.trial168.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::test_open_by_id PASSED [ 4%] 2026-02-22T23:04:38.003 INFO:tasks.workunit.client.0.trial168.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::test_aio_open PASSED [ 4%] 2026-02-22T23:04:38.050 INFO:tasks.workunit.client.0.trial168.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::test_remove_dne PASSED [ 5%] 2026-02-22T23:04:38.069 INFO:tasks.workunit.client.0.trial168.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::test_list_empty PASSED [ 5%] 2026-02-22T23:04:38.293 INFO:tasks.workunit.client.0.trial168.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::test_list PASSED [ 6%] 2026-02-22T23:04:38.467 INFO:tasks.workunit.client.0.trial168.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::test_remove_with_progress PASSED [ 6%] 2026-02-22T23:04:38.693 INFO:tasks.workunit.client.0.trial168.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::test_remove_canceled PASSED [ 7%] 2026-02-22T23:04:38.896 INFO:tasks.workunit.client.0.trial168.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::test_remove_with_progress_except PASSED [ 7%] 2026-02-22T23:04:39.161 INFO:tasks.workunit.client.0.trial168.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::test_rename PASSED [ 8%] 2026-02-22T23:04:39.646 INFO:tasks.workunit.client.0.trial168.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::test_pool_metadata PASSED [ 8%] 2026-02-22T23:04:39.725 INFO:tasks.workunit.client.0.trial168.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::test_config_list PASSED [ 9%] 2026-02-22T23:04:39.776 INFO:tasks.workunit.client.0.trial168.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::test_pool_config_set_and_get_and_remove PASSED [ 9%] 2026-02-22T23:04:39.833 INFO:tasks.workunit.client.0.trial168.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::test_namespaces PASSED [ 10%] 2026-02-22T23:04:41.433 INFO:tasks.workunit.client.0.trial168.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::test_pool_stats PASSED [ 10%] 2026-02-22T23:04:41.447 INFO:tasks.workunit.client.0.trial168.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::test_features_to_string PASSED [ 11%] 2026-02-22T23:04:41.461 INFO:tasks.workunit.client.0.trial168.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::test_features_from_string PASSED [ 11%] 2026-02-22T23:04:41.706 INFO:tasks.workunit.client.0.trial168.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestImage::test_update_features PASSED [ 12%] 2026-02-22T23:04:41.915 INFO:tasks.workunit.client.0.trial168.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestImage::test_create_with_params SKIPPED [ 13%] 2026-02-22T23:04:42.118 INFO:tasks.workunit.client.0.trial168.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestImage::test_id PASSED [ 13%] 2026-02-22T23:04:42.324 INFO:tasks.workunit.client.0.trial168.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestImage::test_block_name_prefix PASSED [ 14%] 2026-02-22T23:04:42.529 INFO:tasks.workunit.client.0.trial168.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestImage::test_data_pool_id PASSED [ 14%] 2026-02-22T23:04:42.738 INFO:tasks.workunit.client.0.trial168.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestImage::test_create_timestamp PASSED [ 15%] 2026-02-22T23:04:42.946 INFO:tasks.workunit.client.0.trial168.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestImage::test_access_timestamp PASSED [ 15%] 2026-02-22T23:04:43.151 INFO:tasks.workunit.client.0.trial168.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestImage::test_modify_timestamp PASSED [ 16%] 2026-02-22T23:04:43.363 INFO:tasks.workunit.client.0.trial168.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestImage::test_invalidate_cache PASSED [ 16%] 2026-02-22T23:04:43.564 INFO:tasks.workunit.client.0.trial168.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestImage::test_stat PASSED [ 17%] 2026-02-22T23:04:43.773 INFO:tasks.workunit.client.0.trial168.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestImage::test_flags PASSED [ 17%] 2026-02-22T23:04:44.012 INFO:tasks.workunit.client.0.trial168.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestImage::test_image_auto_close PASSED [ 18%] 2026-02-22T23:04:44.215 INFO:tasks.workunit.client.0.trial168.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestImage::test_use_after_close PASSED [ 18%] 2026-02-22T23:04:44.429 INFO:tasks.workunit.client.0.trial168.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestImage::test_write PASSED [ 19%] 2026-02-22T23:04:44.639 INFO:tasks.workunit.client.0.trial168.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestImage::test_write_with_fadvise_flags PASSED [ 19%] 2026-02-22T23:04:44.897 INFO:tasks.workunit.client.0.trial168.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestImage::test_write_zeroes PASSED [ 20%] 2026-02-22T23:04:45.114 INFO:tasks.workunit.client.0.trial168.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestImage::test_write_zeroes_thick_provision PASSED [ 20%] 2026-02-22T23:04:45.321 INFO:tasks.workunit.client.0.trial168.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestImage::test_read PASSED [ 21%] 2026-02-22T23:04:45.530 INFO:tasks.workunit.client.0.trial168.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestImage::test_read_with_fadvise_flags PASSED [ 21%] 2026-02-22T23:04:45.818 INFO:tasks.workunit.client.0.trial168.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestImage::test_large_write PASSED [ 22%] 2026-02-22T23:04:46.044 INFO:tasks.workunit.client.0.trial168.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestImage::test_large_read PASSED [ 22%] 2026-02-22T23:04:46.253 INFO:tasks.workunit.client.0.trial168.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestImage::test_write_read PASSED [ 23%] 2026-02-22T23:04:46.457 INFO:tasks.workunit.client.0.trial168.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestImage::test_read_bad_offset PASSED [ 23%] 2026-02-22T23:04:46.668 INFO:tasks.workunit.client.0.trial168.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestImage::test_resize PASSED [ 24%] 2026-02-22T23:04:46.885 INFO:tasks.workunit.client.0.trial168.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestImage::test_resize_allow_shrink_False PASSED [ 25%] 2026-02-22T23:04:50.863 INFO:tasks.workunit.client.0.trial168.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestImage::test_size PASSED [ 25%] 2026-02-22T23:04:51.088 INFO:tasks.workunit.client.0.trial168.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestImage::test_resize_down PASSED [ 26%] 2026-02-22T23:04:51.326 INFO:tasks.workunit.client.0.trial168.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestImage::test_resize_bytes PASSED [ 26%] 2026-02-22T23:04:51.818 INFO:tasks.workunit.client.0.trial168.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestImage::test_copy PASSED [ 27%] 2026-02-22T23:04:52.299 INFO:tasks.workunit.client.0.trial168.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestImage::test_copy2 PASSED [ 27%] 2026-02-22T23:04:52.510 INFO:tasks.workunit.client.0.trial168.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestImage::test_copy3 SKIPPED [ 28%] 2026-02-22T23:04:56.015 INFO:tasks.workunit.client.0.trial168.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestImage::test_deep_copy PASSED [ 28%] 2026-02-22T23:05:01.445 INFO:tasks.workunit.client.0.trial168.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestImage::test_deep_copy_clone_v1_to_v1 PASSED [ 29%] 2026-02-22T23:05:07.458 INFO:tasks.workunit.client.0.trial168.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestImage::test_deep_copy_clone_v1_to_v2 PASSED [ 29%] 2026-02-22T23:05:13.473 INFO:tasks.workunit.client.0.trial168.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestImage::test_deep_copy_clone_v2_to_v1 PASSED [ 30%] 2026-02-22T23:05:19.486 INFO:tasks.workunit.client.0.trial168.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestImage::test_deep_copy_clone_v2_to_v2 PASSED [ 30%] 2026-02-22T23:05:25.509 INFO:tasks.workunit.client.0.trial168.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestImage::test_deep_copy_clone_v1_flatten PASSED [ 31%] 2026-02-22T23:05:31.451 INFO:tasks.workunit.client.0.trial168.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestImage::test_deep_copy_clone_v2_flatten PASSED [ 31%] 2026-02-22T23:05:33.528 INFO:tasks.workunit.client.0.trial168.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestImage::test_create_snap PASSED [ 32%] 2026-02-22T23:05:35.530 INFO:tasks.workunit.client.0.trial168.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestImage::test_create_snap_exists PASSED [ 32%] 2026-02-22T23:05:41.449 INFO:tasks.workunit.client.0.trial168.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestImage::test_create_snap_flags PASSED [ 33%] 2026-02-22T23:05:45.558 INFO:tasks.workunit.client.0.trial168.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestImage::test_list_snaps PASSED [ 33%] 2026-02-22T23:05:47.552 INFO:tasks.workunit.client.0.trial168.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestImage::test_list_snaps_iterator_auto_close PASSED [ 34%] 2026-02-22T23:05:49.560 INFO:tasks.workunit.client.0.trial168.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestImage::test_remove_snap PASSED [ 34%] 2026-02-22T23:05:49.761 INFO:tasks.workunit.client.0.trial168.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestImage::test_remove_snap_not_found PASSED [ 35%] 2026-02-22T23:05:51.450 INFO:tasks.workunit.client.0.trial168.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestImage::test_remove_snap2 PASSED [ 35%] 2026-02-22T23:05:53.570 INFO:tasks.workunit.client.0.trial168.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestImage::test_remove_snap_by_id PASSED [ 36%] 2026-02-22T23:05:55.578 INFO:tasks.workunit.client.0.trial168.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestImage::test_rename_snap PASSED [ 36%] 2026-02-22T23:05:57.586 INFO:tasks.workunit.client.0.trial168.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestImage::test_protect_snap PASSED [ 37%] 2026-02-22T23:05:59.590 INFO:tasks.workunit.client.0.trial168.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestImage::test_snap_exists PASSED [ 38%] 2026-02-22T23:06:01.447 INFO:tasks.workunit.client.0.trial168.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestImage::test_snap_timestamp PASSED [ 38%] 2026-02-22T23:06:09.607 INFO:tasks.workunit.client.0.trial168.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestImage::test_limit_snaps PASSED [ 39%] 2026-02-22T23:06:09.813 INFO:tasks.workunit.client.0.trial168.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestImage::test_remove_with_exclusive_lock SKIPPED [ 39%] 2026-02-22T23:06:11.446 INFO:tasks.workunit.client.0.trial168.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestImage::test_remove_with_snap PASSED [ 40%] 2026-02-22T23:06:11.708 INFO:tasks.workunit.client.0.trial168.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestImage::test_remove_with_watcher PASSED [ 40%] 2026-02-22T23:06:13.611 INFO:tasks.workunit.client.0.trial168.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestImage::test_rollback_to_snap PASSED [ 41%] 2026-02-22T23:06:15.615 INFO:tasks.workunit.client.0.trial168.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestImage::test_rollback_to_snap_sparse PASSED [ 41%] 2026-02-22T23:06:19.625 INFO:tasks.workunit.client.0.trial168.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestImage::test_rollback_with_resize PASSED [ 42%] 2026-02-22T23:06:21.443 INFO:tasks.workunit.client.0.trial168.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestImage::test_set_snap PASSED [ 42%] 2026-02-22T23:06:23.634 INFO:tasks.workunit.client.0.trial168.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestImage::test_set_no_snap PASSED [ 43%] 2026-02-22T23:06:25.637 INFO:tasks.workunit.client.0.trial168.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestImage::test_set_snap_by_id PASSED [ 43%] 2026-02-22T23:06:31.453 INFO:tasks.workunit.client.0.trial168.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestImage::test_snap_get_name PASSED [ 44%] 2026-02-22T23:06:37.673 INFO:tasks.workunit.client.0.trial168.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestImage::test_snap_get_id PASSED [ 44%] 2026-02-22T23:06:39.669 INFO:tasks.workunit.client.0.trial168.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestImage::test_set_snap_sparse PASSED [ 45%] 2026-02-22T23:13:19.509 INFO:tasks.workunit.client.0.trial168.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestImage::test_many_snaps PASSED [ 45%] 2026-02-22T23:13:21.488 INFO:tasks.workunit.client.0.trial168.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestImage::test_set_snap_deleted PASSED [ 46%] 2026-02-22T23:13:25.524 INFO:tasks.workunit.client.0.trial168.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestImage::test_set_snap_recreated PASSED [ 46%] 2026-02-22T23:13:25.750 INFO:tasks.workunit.client.0.trial168.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestImage::test_lock_unlock PASSED [ 47%] 2026-02-22T23:13:26.107 INFO:tasks.workunit.client.0.trial168.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestImage::test_list_lockers PASSED [ 47%] 2026-02-22T23:13:31.485 INFO:tasks.workunit.client.0.trial168.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestImage::test_diff_iterate PASSED [ 48%] 2026-02-22T23:13:33.571 INFO:tasks.workunit.client.0.trial168.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestImage::test_diff_iterate_from_trash_snap PASSED [ 48%] 2026-02-22T23:13:35.551 INFO:tasks.workunit.client.0.trial168.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestImage::test_diff_iterate_exclude_parent PASSED [ 49%] 2026-02-22T23:13:35.761 INFO:tasks.workunit.client.0.trial168.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestImage::test_aio_read PASSED [ 50%] 2026-02-22T23:13:35.969 INFO:tasks.workunit.client.0.trial168.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestImage::test_aio_write PASSED [ 50%] 2026-02-22T23:13:36.175 INFO:tasks.workunit.client.0.trial168.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestImage::test_aio_discard PASSED [ 51%] 2026-02-22T23:13:36.390 INFO:tasks.workunit.client.0.trial168.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestImage::test_aio_write_zeroes PASSED [ 51%] 2026-02-22T23:13:36.597 INFO:tasks.workunit.client.0.trial168.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestImage::test_aio_flush PASSED [ 52%] 2026-02-22T23:13:37.978 INFO:tasks.workunit.client.0.trial168.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestImage::test_metadata PASSED [ 52%] 2026-02-22T23:13:38.181 INFO:tasks.workunit.client.0.trial168.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestImage::test_watchers_list PASSED [ 53%] 2026-02-22T23:13:38.468 INFO:tasks.workunit.client.0.trial168.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestImage::test_config_list PASSED [ 53%] 2026-02-22T23:13:38.748 INFO:tasks.workunit.client.0.trial168.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestImage::test_image_config_set_and_get_and_remove PASSED [ 54%] 2026-02-22T23:13:38.963 INFO:tasks.workunit.client.0.trial168.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestImage::test_sparsify PASSED [ 54%] 2026-02-22T23:13:48.471 INFO:tasks.workunit.client.0.trial168.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestImage::test_encryption_luks1 PASSED [ 55%] 2026-02-22T23:14:04.400 INFO:tasks.workunit.client.0.trial168.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestImage::test_encryption_luks2 PASSED [ 55%] 2026-02-22T23:14:04.637 INFO:tasks.workunit.client.0.trial168.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestImageId::test_read PASSED [ 56%] 2026-02-22T23:14:04.878 INFO:tasks.workunit.client.0.trial168.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestImageId::test_write PASSED [ 56%] 2026-02-22T23:14:05.127 INFO:tasks.workunit.client.0.trial168.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestImageId::test_resize PASSED [ 57%] 2026-02-22T23:14:08.500 INFO:tasks.workunit.client.0.trial168.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestClone::test_with_params PASSED [ 57%] 2026-02-22T23:14:12.509 INFO:tasks.workunit.client.0.trial168.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestClone::test_with_params2 PASSED [ 58%] 2026-02-22T23:14:14.514 INFO:tasks.workunit.client.0.trial168.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestClone::test_with_params3 SKIPPED [ 58%] 2026-02-22T23:14:18.528 INFO:tasks.workunit.client.0.trial168.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestClone::test_stripe_unit_and_count PASSED [ 59%] 2026-02-22T23:14:20.533 INFO:tasks.workunit.client.0.trial168.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestClone::test_clone_format PASSED [ 59%] 2026-02-22T23:14:24.545 INFO:tasks.workunit.client.0.trial168.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestClone::test_unprotected PASSED [ 60%] 2026-02-22T23:14:31.497 INFO:tasks.workunit.client.0.trial168.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestClone::test_unprotect_with_children PASSED [ 60%] 2026-02-22T23:14:37.511 INFO:tasks.workunit.client.0.trial168.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestClone::test_clone_by_snap_id PASSED [ 61%] 2026-02-22T23:14:39.512 INFO:tasks.workunit.client.0.trial168.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestClone::test_stat PASSED [ 61%] 2026-02-22T23:14:41.519 INFO:tasks.workunit.client.0.trial168.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestClone::test_resize_stat PASSED [ 62%] 2026-02-22T23:14:43.528 INFO:tasks.workunit.client.0.trial168.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestClone::test_resize_io PASSED [ 63%] 2026-02-22T23:14:45.525 INFO:tasks.workunit.client.0.trial168.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestClone::test_read PASSED [ 63%] 2026-02-22T23:14:47.536 INFO:tasks.workunit.client.0.trial168.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestClone::test_write PASSED [ 64%] 2026-02-22T23:15:02.543 INFO:tasks.workunit.client.0.trial168.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestClone::test_list_children PASSED [ 64%] 2026-02-22T23:15:06.556 INFO:tasks.workunit.client.0.trial168.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestClone::test_flatten_errors PASSED [ 65%] 2026-02-22T23:15:09.563 INFO:tasks.workunit.client.0.trial168.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestClone::test_flatten_basic PASSED [ 65%] 2026-02-22T23:15:12.573 INFO:tasks.workunit.client.0.trial168.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestClone::test_flatten_smaller_order PASSED [ 66%] 2026-02-22T23:15:15.578 INFO:tasks.workunit.client.0.trial168.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestClone::test_flatten_larger_order PASSED [ 66%] 2026-02-22T23:15:17.585 INFO:tasks.workunit.client.0.trial168.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestClone::test_flatten_drops_cache PASSED [ 67%] 2026-02-22T23:15:21.494 INFO:tasks.workunit.client.0.trial168.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestClone::test_flatten_multi_level PASSED [ 67%] 2026-02-22T23:15:23.601 INFO:tasks.workunit.client.0.trial168.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestClone::test_flatten_with_progress PASSED [ 68%] 2026-02-22T23:15:27.612 INFO:tasks.workunit.client.0.trial168.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestClone::test_resize_flatten_multi_level PASSED [ 68%] 2026-02-22T23:15:31.494 INFO:tasks.workunit.client.0.trial168.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestClone::test_trash_snapshot PASSED [ 69%] 2026-02-22T23:15:39.631 INFO:tasks.workunit.client.0.trial168.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestClone::test_encryption_luks1 PASSED [ 69%] 2026-02-22T23:15:53.500 INFO:tasks.workunit.client.0.trial168.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestClone::test_encryption_luks2 PASSED [ 70%] 2026-02-22T23:15:53.511 INFO:tasks.workunit.client.0.trial168.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestExclusiveLock::test_ownership SKIPPED [ 70%] 2026-02-22T23:15:53.522 INFO:tasks.workunit.client.0.trial168.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestExclusiveLock::test_snapshot_leadership SKIPPED [ 71%] 2026-02-22T23:15:53.533 INFO:tasks.workunit.client.0.trial168.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestExclusiveLock::test_read_only_leadership SKIPPED [ 71%] 2026-02-22T23:15:53.544 INFO:tasks.workunit.client.0.trial168.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestExclusiveLock::test_follower_flatten SKIPPED [ 72%] 2026-02-22T23:15:53.555 INFO:tasks.workunit.client.0.trial168.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestExclusiveLock::test_follower_resize SKIPPED [ 72%] 2026-02-22T23:15:53.566 INFO:tasks.workunit.client.0.trial168.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestExclusiveLock::test_follower_snap_create SKIPPED [ 73%] 2026-02-22T23:15:53.577 INFO:tasks.workunit.client.0.trial168.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestExclusiveLock::test_follower_snap_rollback SKIPPED [ 73%] 2026-02-22T23:15:53.588 INFO:tasks.workunit.client.0.trial168.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestExclusiveLock::test_follower_discard SKIPPED [ 74%] 2026-02-22T23:15:53.598 INFO:tasks.workunit.client.0.trial168.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestExclusiveLock::test_follower_write SKIPPED [ 75%] 2026-02-22T23:15:53.609 INFO:tasks.workunit.client.0.trial168.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestExclusiveLock::test_acquire_release_lock SKIPPED [ 75%] 2026-02-22T23:15:53.621 INFO:tasks.workunit.client.0.trial168.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestExclusiveLock::test_break_lock SKIPPED [ 76%] 2026-02-22T23:15:53.851 INFO:tasks.workunit.client.0.trial168.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestMirroring::test_uuid PASSED [ 76%] 2026-02-22T23:15:54.084 INFO:tasks.workunit.client.0.trial168.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestMirroring::test_site_name PASSED [ 77%] 2026-02-22T23:15:54.541 INFO:tasks.workunit.client.0.trial168.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestMirroring::test_mirror_peer_bootstrap PASSED [ 77%] 2026-02-22T23:15:54.811 INFO:tasks.workunit.client.0.trial168.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestMirroring::test_mirror_peer PASSED [ 78%] 2026-02-22T23:15:55.026 INFO:tasks.workunit.client.0.trial168.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestMirroring::test_mirror_image SKIPPED [ 78%] 2026-02-22T23:15:55.242 INFO:tasks.workunit.client.0.trial168.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestMirroring::test_mirror_image_status SKIPPED [ 79%] 2026-02-22T23:16:01.428 INFO:tasks.workunit.client.0.trial168.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestMirroring::test_mirror_image_create_snapshot PASSED [ 79%] 2026-02-22T23:16:05.446 INFO:tasks.workunit.client.0.trial168.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestMirroring::test_aio_mirror_image_create_snapshot PASSED [ 80%] 2026-02-22T23:16:05.893 INFO:tasks.workunit.client.0.trial168.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestTrash::test_move PASSED [ 80%] 2026-02-22T23:16:06.488 INFO:tasks.workunit.client.0.trial168.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestTrash::test_purge PASSED [ 81%] 2026-02-22T23:16:06.934 INFO:tasks.workunit.client.0.trial168.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestTrash::test_remove_denied PASSED [ 81%] 2026-02-22T23:16:07.376 INFO:tasks.workunit.client.0.trial168.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestTrash::test_remove PASSED [ 82%] 2026-02-22T23:16:07.821 INFO:tasks.workunit.client.0.trial168.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestTrash::test_remove_with_progress PASSED [ 82%] 2026-02-22T23:16:08.274 INFO:tasks.workunit.client.0.trial168.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestTrash::test_get PASSED [ 83%] 2026-02-22T23:16:08.855 INFO:tasks.workunit.client.0.trial168.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestTrash::test_list PASSED [ 83%] 2026-02-22T23:16:09.395 INFO:tasks.workunit.client.0.trial168.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestTrash::test_restore PASSED [ 84%] 2026-02-22T23:16:09.422 INFO:tasks.workunit.client.0.trial168.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::test_create_group PASSED [ 84%] 2026-02-22T23:16:09.450 INFO:tasks.workunit.client.0.trial168.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::test_rename_group PASSED [ 85%] 2026-02-22T23:16:09.461 INFO:tasks.workunit.client.0.trial168.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::test_list_groups_empty PASSED [ 85%] 2026-02-22T23:16:09.480 INFO:tasks.workunit.client.0.trial168.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::test_list_groups PASSED [ 86%] 2026-02-22T23:16:09.499 INFO:tasks.workunit.client.0.trial168.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::test_list_groups_after_removed PASSED [ 86%] 2026-02-22T23:16:09.733 INFO:tasks.workunit.client.0.trial168.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestGroups::test_group_image_add PASSED [ 87%] 2026-02-22T23:16:09.950 INFO:tasks.workunit.client.0.trial168.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestGroups::test_group_image_list_empty PASSED [ 88%] 2026-02-22T23:16:10.177 INFO:tasks.workunit.client.0.trial168.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestGroups::test_group_image_list PASSED [ 88%] 2026-02-22T23:16:10.438 INFO:tasks.workunit.client.0.trial168.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestGroups::test_group_image_list_move_to_trash PASSED [ 89%] 2026-02-22T23:16:10.886 INFO:tasks.workunit.client.0.trial168.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestGroups::test_group_image_list_remove PASSED [ 89%] 2026-02-22T23:16:14.448 INFO:tasks.workunit.client.0.trial168.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestGroups::test_group_image_many_images PASSED [ 90%] 2026-02-22T23:16:14.746 INFO:tasks.workunit.client.0.trial168.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestGroups::test_group_image_remove PASSED [ 90%] 2026-02-22T23:16:14.984 INFO:tasks.workunit.client.0.trial168.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestGroups::test_group_snap PASSED [ 91%] 2026-02-22T23:16:15.222 INFO:tasks.workunit.client.0.trial168.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestGroups::test_group_snap_flags PASSED [ 91%] 2026-02-22T23:16:15.537 INFO:tasks.workunit.client.0.trial168.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestGroups::test_group_snap_list_many PASSED [ 92%] 2026-02-22T23:16:17.522 INFO:tasks.workunit.client.0.trial168.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestGroups::test_group_snap_namespace PASSED [ 92%] 2026-02-22T23:16:17.749 INFO:tasks.workunit.client.0.trial168.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestGroups::test_group_snap_rename PASSED [ 93%] 2026-02-22T23:16:19.550 INFO:tasks.workunit.client.0.trial168.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestGroups::test_group_snap_clone PASSED [ 93%] 2026-02-22T23:16:21.721 INFO:tasks.workunit.client.0.trial168.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestGroups::test_group_snap_clone_flatten PASSED [ 94%] 2026-02-22T23:16:23.546 INFO:tasks.workunit.client.0.trial168.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestGroups::test_group_snap_diff_iterate PASSED [ 94%] 2026-02-22T23:16:30.818 INFO:tasks.workunit.client.0.trial168.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestGroups::test_group_snap_rollback PASSED [ 95%] 2026-02-22T23:16:31.689 INFO:tasks.workunit.client.0.trial168.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestMigration::test_migration PASSED [ 95%] 2026-02-22T23:16:34.546 INFO:tasks.workunit.client.0.trial168.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestMigration::test_migration_clone_v1_to_v1 PASSED [ 96%] 2026-02-22T23:16:37.552 INFO:tasks.workunit.client.0.trial168.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestMigration::test_migration_clone_v1_to_v2 PASSED [ 96%] 2026-02-22T23:16:40.560 INFO:tasks.workunit.client.0.trial168.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestMigration::test_migration_clone_v2_to_v1 PASSED [ 97%] 2026-02-22T23:16:43.570 INFO:tasks.workunit.client.0.trial168.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestMigration::test_migration_clone_v2_to_v2 PASSED [ 97%] 2026-02-22T23:16:47.666 INFO:tasks.workunit.client.0.trial168.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestMigration::test_migration_import PASSED [ 98%] 2026-02-22T23:16:48.271 INFO:tasks.workunit.client.0.trial168.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestMigration::test_migration_with_progress PASSED [ 98%] 2026-02-22T23:16:48.742 INFO:tasks.workunit.client.0.trial168.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestMigration::test_migrate_abort PASSED [ 99%] 2026-02-22T23:16:49.454 INFO:tasks.workunit.client.0.trial168.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestMigration::test_migrate_abort_with_progress PASSED [100%] 2026-02-22T23:16:49.454 INFO:tasks.workunit.client.0.trial168.stdout: 2026-02-22T23:16:49.454 INFO:tasks.workunit.client.0.trial168.stdout:=============================== warnings summary =============================== 2026-02-22T23:16:49.454 INFO:tasks.workunit.client.0.trial168.stdout:test_rbd.py::test_remove_with_progress_except 2026-02-22T23:16:49.455 INFO:tasks.workunit.client.0.trial168.stdout: /usr/lib/python3.9/site-packages/_pytest/unraisableexception.py:78: PytestUnraisableExceptionWarning: Exception ignored in: 'rbd.progress_callback' 2026-02-22T23:16:49.455 INFO:tasks.workunit.client.0.trial168.stdout: 2026-02-22T23:16:49.455 INFO:tasks.workunit.client.0.trial168.stdout: Traceback (most recent call last): 2026-02-22T23:16:49.455 INFO:tasks.workunit.client.0.trial168.stdout: File "/home/ubuntu/cephtest/clone.client.0/src/test/pybind/test_rbd.py", line 428, in progress_cb 2026-02-22T23:16:49.455 INFO:tasks.workunit.client.0.trial168.stdout: raise Exception() 2026-02-22T23:16:49.455 INFO:tasks.workunit.client.0.trial168.stdout: Exception 2026-02-22T23:16:49.455 INFO:tasks.workunit.client.0.trial168.stdout: 2026-02-22T23:16:49.455 INFO:tasks.workunit.client.0.trial168.stdout: warnings.warn(pytest.PytestUnraisableExceptionWarning(msg)) 2026-02-22T23:16:49.455 INFO:tasks.workunit.client.0.trial168.stdout: 2026-02-22T23:16:49.455 INFO:tasks.workunit.client.0.trial168.stdout:-- Docs: https://docs.pytest.org/en/stable/warnings.html 2026-02-22T23:16:49.455 INFO:tasks.workunit.client.0.trial168.stdout:============ 175 passed, 17 skipped, 1 warning in 790.47s (0:13:10) ============ 2026-02-22T23:16:49.670 INFO:tasks.workunit.client.0.trial168.stderr:==40121== 2026-02-22T23:16:49.672 INFO:tasks.workunit.client.0.trial168.stderr:==40121== HEAP SUMMARY: 2026-02-22T23:16:49.672 INFO:tasks.workunit.client.0.trial168.stderr:==40121== in use at exit: 9,368,075 bytes in 85,274 blocks 2026-02-22T23:16:49.672 INFO:tasks.workunit.client.0.trial168.stderr:==40121== total heap usage: 28,594,861 allocs, 28,509,587 frees, 5,711,928,774 bytes allocated 2026-02-22T23:16:49.673 INFO:tasks.workunit.client.0.trial168.stderr:==40121== 2026-02-22T23:16:49.823 INFO:tasks.workunit.client.0.trial168.stderr:==40121== 488 bytes in 1 blocks are possibly lost in loss record 9,672 of 11,614 2026-02-22T23:16:49.823 INFO:tasks.workunit.client.0.trial168.stderr:==40121== at 0x48536B8: realloc (vg_replace_malloc.c:1804) 2026-02-22T23:16:49.823 INFO:tasks.workunit.client.0.trial168.stderr:==40121== by 0x49C39EC: _PyObject_GC_Resize (gcmodule.c:2339) 2026-02-22T23:16:49.823 INFO:tasks.workunit.client.0.trial168.stderr:==40121== by 0x49AAB3F: frame_alloc (frameobject.c:805) 2026-02-22T23:16:49.824 INFO:tasks.workunit.client.0.trial168.stderr:==40121== by 0x49AAB3F: _PyFrame_New_NoTrack (frameobject.c:888) 2026-02-22T23:16:49.824 INFO:tasks.workunit.client.0.trial168.stderr:==40121== by 0x49AAB3F: function_code_fastcall (call.c:319) 2026-02-22T23:16:49.824 INFO:tasks.workunit.client.0.trial168.stderr:==40121== by 0x499CD18: do_call_core (ceval.c:5125) 2026-02-22T23:16:49.824 INFO:tasks.workunit.client.0.trial168.stderr:==40121== by 0x499CD18: _PyEval_EvalFrameDefault (ceval.c:3582) 2026-02-22T23:16:49.824 INFO:tasks.workunit.client.0.trial168.stderr:==40121== by 0x4997EE8: _PyEval_EvalFrame (pycore_ceval.h:40) 2026-02-22T23:16:49.824 INFO:tasks.workunit.client.0.trial168.stderr:==40121== by 0x4997EE8: _PyEval_EvalCode (ceval.c:4329) 2026-02-22T23:16:49.824 INFO:tasks.workunit.client.0.trial168.stderr:==40121== by 0x49AA5D4: _PyFunction_Vectorcall (call.c:396) 2026-02-22T23:16:49.824 INFO:tasks.workunit.client.0.trial168.stderr:==40121== by 0x499CD18: do_call_core (ceval.c:5125) 2026-02-22T23:16:49.824 INFO:tasks.workunit.client.0.trial168.stderr:==40121== by 0x499CD18: _PyEval_EvalFrameDefault (ceval.c:3582) 2026-02-22T23:16:49.824 INFO:tasks.workunit.client.0.trial168.stderr:==40121== by 0x4997EE8: _PyEval_EvalFrame (pycore_ceval.h:40) 2026-02-22T23:16:49.824 INFO:tasks.workunit.client.0.trial168.stderr:==40121== by 0x4997EE8: _PyEval_EvalCode (ceval.c:4329) 2026-02-22T23:16:49.825 INFO:tasks.workunit.client.0.trial168.stderr:==40121== by 0x49AA5D4: _PyFunction_Vectorcall (call.c:396) 2026-02-22T23:16:49.825 INFO:tasks.workunit.client.0.trial168.stderr:==40121== by 0x49B8C94: _PyObject_VectorcallTstate (abstract.h:118) 2026-02-22T23:16:49.825 INFO:tasks.workunit.client.0.trial168.stderr:==40121== by 0x49B8C94: method_vectorcall (classobject.c:83) 2026-02-22T23:16:49.825 INFO:tasks.workunit.client.0.trial168.stderr:==40121== by 0x499CD18: do_call_core (ceval.c:5125) 2026-02-22T23:16:49.825 INFO:tasks.workunit.client.0.trial168.stderr:==40121== by 0x499CD18: _PyEval_EvalFrameDefault (ceval.c:3582) 2026-02-22T23:16:49.825 INFO:tasks.workunit.client.0.trial168.stderr:==40121== by 0x4997EE8: _PyEval_EvalFrame (pycore_ceval.h:40) 2026-02-22T23:16:49.825 INFO:tasks.workunit.client.0.trial168.stderr:==40121== by 0x4997EE8: _PyEval_EvalCode (ceval.c:4329) 2026-02-22T23:16:49.825 INFO:tasks.workunit.client.0.trial168.stderr:==40121== 2026-02-22T23:16:49.826 INFO:tasks.workunit.client.0.trial168.stderr:==40121== 528 bytes in 1 blocks are possibly lost in loss record 9,796 of 11,614 2026-02-22T23:16:49.826 INFO:tasks.workunit.client.0.trial168.stderr:==40121== at 0x484B82F: malloc (vg_replace_malloc.c:447) 2026-02-22T23:16:49.826 INFO:tasks.workunit.client.0.trial168.stderr:==40121== by 0x4018058: malloc (rtld-malloc.h:56) 2026-02-22T23:16:49.826 INFO:tasks.workunit.client.0.trial168.stderr:==40121== by 0x4018058: _dl_resize_dtv (dl-tls.c:533) 2026-02-22T23:16:49.826 INFO:tasks.workunit.client.0.trial168.stderr:==40121== by 0x4018A00: _dl_update_slotinfo (dl-tls.c:871) 2026-02-22T23:16:49.827 INFO:tasks.workunit.client.0.trial168.stderr:==40121== by 0x4018B2B: update_get_addr (dl-tls.c:989) 2026-02-22T23:16:49.827 INFO:tasks.workunit.client.0.trial168.stderr:==40121== by 0x401BEEB: __tls_get_addr (tls_get_addr.S:55) 2026-02-22T23:16:49.827 INFO:tasks.workunit.client.0.trial168.stderr:==40121== by 0xC7AA78B: _gnutls_fips_mode_enabled (fips.c:84) 2026-02-22T23:16:49.827 INFO:tasks.workunit.client.0.trial168.stderr:==40121== by 0xC7AA78B: _gnutls_rnd_preinit (random.c:162) 2026-02-22T23:16:49.827 INFO:tasks.workunit.client.0.trial168.stderr:==40121== by 0xC7AA78B: _gnutls_global_init (global.c:323) 2026-02-22T23:16:49.827 INFO:tasks.workunit.client.0.trial168.stderr:==40121== by 0xC7737A4: lib_init.lto_priv.0 (global.c:550) 2026-02-22T23:16:49.827 INFO:tasks.workunit.client.0.trial168.stderr:==40121== by 0x400B4FD: call_init (dl-init.c:70) 2026-02-22T23:16:49.827 INFO:tasks.workunit.client.0.trial168.stderr:==40121== by 0x400B4FD: call_init (dl-init.c:26) 2026-02-22T23:16:49.827 INFO:tasks.workunit.client.0.trial168.stderr:==40121== by 0x400B5EB: _dl_init (dl-init.c:117) 2026-02-22T23:16:49.827 INFO:tasks.workunit.client.0.trial168.stderr:==40121== by 0x40074D5: _dl_catch_exception (dl-catch.c:211) 2026-02-22T23:16:49.827 INFO:tasks.workunit.client.0.trial168.stderr:==40121== by 0x4012095: dl_open_worker (dl-open.c:827) 2026-02-22T23:16:49.827 INFO:tasks.workunit.client.0.trial168.stderr:==40121== by 0x4007444: _dl_catch_exception (dl-catch.c:237) 2026-02-22T23:16:49.828 INFO:tasks.workunit.client.0.trial168.stderr:==40121== 2026-02-22T23:16:49.833 INFO:tasks.workunit.client.0.trial168.stderr:==40121== 4,960 bytes in 9 blocks are possibly lost in loss record 11,347 of 11,614 2026-02-22T23:16:49.833 INFO:tasks.workunit.client.0.trial168.stderr:==40121== at 0x48536B8: realloc (vg_replace_malloc.c:1804) 2026-02-22T23:16:49.833 INFO:tasks.workunit.client.0.trial168.stderr:==40121== by 0x49C39EC: _PyObject_GC_Resize (gcmodule.c:2339) 2026-02-22T23:16:49.833 INFO:tasks.workunit.client.0.trial168.stderr:==40121== by 0x49AAB3F: frame_alloc (frameobject.c:805) 2026-02-22T23:16:49.833 INFO:tasks.workunit.client.0.trial168.stderr:==40121== by 0x49AAB3F: _PyFrame_New_NoTrack (frameobject.c:888) 2026-02-22T23:16:49.833 INFO:tasks.workunit.client.0.trial168.stderr:==40121== by 0x49AAB3F: function_code_fastcall (call.c:319) 2026-02-22T23:16:49.833 INFO:tasks.workunit.client.0.trial168.stderr:==40121== by 0x499CD18: do_call_core (ceval.c:5125) 2026-02-22T23:16:49.834 INFO:tasks.workunit.client.0.trial168.stderr:==40121== by 0x499CD18: _PyEval_EvalFrameDefault (ceval.c:3582) 2026-02-22T23:16:49.834 INFO:tasks.workunit.client.0.trial168.stderr:==40121== by 0x4997EE8: _PyEval_EvalFrame (pycore_ceval.h:40) 2026-02-22T23:16:49.834 INFO:tasks.workunit.client.0.trial168.stderr:==40121== by 0x4997EE8: _PyEval_EvalCode (ceval.c:4329) 2026-02-22T23:16:49.834 INFO:tasks.workunit.client.0.trial168.stderr:==40121== by 0x49AA5D4: _PyFunction_Vectorcall (call.c:396) 2026-02-22T23:16:49.834 INFO:tasks.workunit.client.0.trial168.stderr:==40121== by 0x49B8C94: _PyObject_VectorcallTstate (abstract.h:118) 2026-02-22T23:16:49.834 INFO:tasks.workunit.client.0.trial168.stderr:==40121== by 0x49B8C94: method_vectorcall (classobject.c:83) 2026-02-22T23:16:49.834 INFO:tasks.workunit.client.0.trial168.stderr:==40121== by 0x499CD18: do_call_core (ceval.c:5125) 2026-02-22T23:16:49.834 INFO:tasks.workunit.client.0.trial168.stderr:==40121== by 0x499CD18: _PyEval_EvalFrameDefault (ceval.c:3582) 2026-02-22T23:16:49.834 INFO:tasks.workunit.client.0.trial168.stderr:==40121== by 0x4997EE8: _PyEval_EvalFrame (pycore_ceval.h:40) 2026-02-22T23:16:49.834 INFO:tasks.workunit.client.0.trial168.stderr:==40121== by 0x4997EE8: _PyEval_EvalCode (ceval.c:4329) 2026-02-22T23:16:49.834 INFO:tasks.workunit.client.0.trial168.stderr:==40121== by 0x49AA5D4: _PyFunction_Vectorcall (call.c:396) 2026-02-22T23:16:49.834 INFO:tasks.workunit.client.0.trial168.stderr:==40121== by 0x499CD18: do_call_core (ceval.c:5125) 2026-02-22T23:16:49.835 INFO:tasks.workunit.client.0.trial168.stderr:==40121== by 0x499CD18: _PyEval_EvalFrameDefault (ceval.c:3582) 2026-02-22T23:16:49.835 INFO:tasks.workunit.client.0.trial168.stderr:==40121== by 0x4997EE8: _PyEval_EvalFrame (pycore_ceval.h:40) 2026-02-22T23:16:49.835 INFO:tasks.workunit.client.0.trial168.stderr:==40121== by 0x4997EE8: _PyEval_EvalCode (ceval.c:4329) 2026-02-22T23:16:49.835 INFO:tasks.workunit.client.0.trial168.stderr:==40121== 2026-02-22T23:16:49.835 INFO:tasks.workunit.client.0.trial168.stderr:==40121== LEAK SUMMARY: 2026-02-22T23:16:49.836 INFO:tasks.workunit.client.0.trial168.stderr:==40121== definitely lost: 0 bytes in 0 blocks 2026-02-22T23:16:49.836 INFO:tasks.workunit.client.0.trial168.stderr:==40121== indirectly lost: 0 bytes in 0 blocks 2026-02-22T23:16:49.836 INFO:tasks.workunit.client.0.trial168.stderr:==40121== possibly lost: 5,976 bytes in 11 blocks 2026-02-22T23:16:49.836 INFO:tasks.workunit.client.0.trial168.stderr:==40121== still reachable: 508,979 bytes in 5,050 blocks 2026-02-22T23:16:49.836 INFO:tasks.workunit.client.0.trial168.stderr:==40121== suppressed: 8,853,120 bytes in 80,213 blocks 2026-02-22T23:16:49.836 INFO:tasks.workunit.client.0.trial168.stderr:==40121== Reachable blocks (those to which a pointer was found) are not shown. 2026-02-22T23:16:49.836 INFO:tasks.workunit.client.0.trial168.stderr:==40121== To see them, rerun with: --leak-check=full --show-leak-kinds=all 2026-02-22T23:16:49.836 INFO:tasks.workunit.client.0.trial168.stderr:==40121== 2026-02-22T23:16:49.836 INFO:tasks.workunit.client.0.trial168.stderr:==40121== For lists of detected and suppressed errors, rerun with: -s 2026-02-22T23:16:49.836 INFO:tasks.workunit.client.0.trial168.stderr:==40121== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 3214 from 6) 2026-02-22T23:16:49.845 INFO:tasks.workunit.client.0.trial168.stderr:+ exit 0 2026-02-22T23:16:49.846 INFO:teuthology.orchestra.run:Running command with timeout 3600 2026-02-22T23:16:49.846 DEBUG:teuthology.orchestra.run.trial168:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2026-02-22T23:16:49.913 INFO:tasks.workunit:Stopping ['rbd/test_librbd_python.sh'] on client.0... 2026-02-22T23:16:49.914 DEBUG:teuthology.orchestra.run.trial168:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2026-02-22T23:16:50.252 DEBUG:teuthology.parallel:result is None 2026-02-22T23:16:50.252 DEBUG:teuthology.orchestra.run.trial168:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2026-02-22T23:16:50.277 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2026-02-22T23:16:50.277 DEBUG:teuthology.orchestra.run.trial168:> rmdir -- /home/ubuntu/cephtest/mnt.0 2026-02-22T23:16:50.334 INFO:tasks.workunit:Deleted artificial mount point /home/ubuntu/cephtest/mnt.0/client.0 2026-02-22T23:16:50.334 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2026-02-22T23:16:50.348 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2026-02-22T23:16:50.348 DEBUG:teuthology.orchestra.run.trial168:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T23:16:50.532 INFO:teuthology.orchestra.run.trial168.stdout: 2026-02-22T23:16:50.532 INFO:teuthology.orchestra.run.trial168.stderr:dumped all 2026-02-22T23:16:50.540 INFO:teuthology.orchestra.run.trial168.stdout:{"pg_ready":true,"pg_map":{"version":1089,"stamp":"2026-02-22T23:16:49.348195+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":103,"num_write_kb":1630,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":75,"ondisk_log_size":75,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":177328,"kb_used_data":17324,"kb_used_omap":4,"kb_used_meta":159931,"kb_avail":282938192,"statfs":{"total":289910292480,"available":289728708608,"internally_reserved":0,"allocated":17739776,"data_stored":8976414,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4782,"internal_metadata":163769682},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-43,"num_objects":-9,"num_object_clones":0,"num_object_copies":-18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-9,"num_whiteouts":0,"num_read":-365982,"num_read_kb":-295194,"num_write":-7769,"num_write_kb":-78746,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.616678"},"pg_stats":[{"pgid":"1.0","version":"10'72","reported_seq":1428,"reported_epoch":671,"state":"active+clean","last_fresh":"2026-02-22T23:16:45.419414+0000","last_change":"2026-02-22T23:02:56.646381+0000","last_active":"2026-02-22T23:16:45.419414+0000","last_peered":"2026-02-22T23:16:45.419414+0000","last_clean":"2026-02-22T23:16:45.419414+0000","last_became_active":"2026-02-22T23:02:56.645752+0000","last_became_peered":"2026-02-22T23:02:56.645752+0000","last_unstale":"2026-02-22T23:16:45.419414+0000","last_undegraded":"2026-02-22T23:16:45.419414+0000","last_fullsized":"2026-02-22T23:16:45.419414+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T23:02:55.640202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T23:02:55.640202+0000","last_clean_scrub_stamp":"2026-02-22T23:02:55.640202+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T05:06:17.108040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"13'1","reported_seq":1338,"reported_epoch":671,"state":"active+clean","last_fresh":"2026-02-22T23:16:45.419942+0000","last_change":"2026-02-22T23:03:03.657709+0000","last_active":"2026-02-22T23:16:45.419942+0000","last_peered":"2026-02-22T23:16:45.419942+0000","last_clean":"2026-02-22T23:16:45.419942+0000","last_became_active":"2026-02-22T23:03:01.654744+0000","last_became_peered":"2026-02-22T23:03:01.654744+0000","last_unstale":"2026-02-22T23:16:45.419942+0000","last_undegraded":"2026-02-22T23:16:45.419942+0000","last_fullsized":"2026-02-22T23:16:45.419942+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T23:03:00.647409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T23:03:00.647409+0000","last_clean_scrub_stamp":"2026-02-22T23:03:00.647409+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T08:30:18.919810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00079898700000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":1297,"reported_epoch":671,"state":"active+clean","last_fresh":"2026-02-22T23:16:45.422453+0000","last_change":"2026-02-22T23:03:04.197929+0000","last_active":"2026-02-22T23:16:45.422453+0000","last_peered":"2026-02-22T23:16:45.422453+0000","last_clean":"2026-02-22T23:16:45.422453+0000","last_became_active":"2026-02-22T23:03:01.655068+0000","last_became_peered":"2026-02-22T23:03:01.655068+0000","last_unstale":"2026-02-22T23:16:45.422453+0000","last_undegraded":"2026-02-22T23:16:45.422453+0000","last_fullsized":"2026-02-22T23:16:45.422453+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T23:03:00.647409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T23:03:00.647409+0000","last_clean_scrub_stamp":"2026-02-22T23:03:00.647409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T00:00:11.473581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00033282899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":1297,"reported_epoch":671,"state":"active+clean","last_fresh":"2026-02-22T23:16:45.422218+0000","last_change":"2026-02-22T23:03:04.197991+0000","last_active":"2026-02-22T23:16:45.422218+0000","last_peered":"2026-02-22T23:16:45.422218+0000","last_clean":"2026-02-22T23:16:45.422218+0000","last_became_active":"2026-02-22T23:03:01.655046+0000","last_became_peered":"2026-02-22T23:03:01.655046+0000","last_unstale":"2026-02-22T23:16:45.422218+0000","last_undegraded":"2026-02-22T23:16:45.422218+0000","last_fullsized":"2026-02-22T23:16:45.422218+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T23:03:00.647409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T23:03:00.647409+0000","last_clean_scrub_stamp":"2026-02-22T23:03:00.647409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T05:07:31.304712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00038590800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"15'2","reported_seq":1337,"reported_epoch":671,"state":"active+clean","last_fresh":"2026-02-22T23:16:45.426754+0000","last_change":"2026-02-22T23:03:03.656168+0000","last_active":"2026-02-22T23:16:45.426754+0000","last_peered":"2026-02-22T23:16:45.426754+0000","last_clean":"2026-02-22T23:16:45.426754+0000","last_became_active":"2026-02-22T23:03:01.655020+0000","last_became_peered":"2026-02-22T23:03:01.655020+0000","last_unstale":"2026-02-22T23:16:45.426754+0000","last_undegraded":"2026-02-22T23:16:45.426754+0000","last_fullsized":"2026-02-22T23:16:45.426754+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T23:03:00.647409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T23:03:00.647409+0000","last_clean_scrub_stamp":"2026-02-22T23:03:00.647409+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T01:25:19.891118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000724969,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":1337,"reported_epoch":671,"state":"active+clean","last_fresh":"2026-02-22T23:16:45.420161+0000","last_change":"2026-02-22T23:03:03.657389+0000","last_active":"2026-02-22T23:16:45.420161+0000","last_peered":"2026-02-22T23:16:45.420161+0000","last_clean":"2026-02-22T23:16:45.420161+0000","last_became_active":"2026-02-22T23:03:01.654989+0000","last_became_peered":"2026-02-22T23:03:01.654989+0000","last_unstale":"2026-02-22T23:16:45.420161+0000","last_undegraded":"2026-02-22T23:16:45.420161+0000","last_fullsized":"2026-02-22T23:16:45.420161+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T23:03:00.647409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T23:03:00.647409+0000","last_clean_scrub_stamp":"2026-02-22T23:03:00.647409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T01:06:44.014188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000472108,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":1337,"reported_epoch":671,"state":"active+clean","last_fresh":"2026-02-22T23:16:45.419958+0000","last_change":"2026-02-22T23:03:03.657408+0000","last_active":"2026-02-22T23:16:45.419958+0000","last_peered":"2026-02-22T23:16:45.419958+0000","last_clean":"2026-02-22T23:16:45.419958+0000","last_became_active":"2026-02-22T23:03:01.655252+0000","last_became_peered":"2026-02-22T23:03:01.655252+0000","last_unstale":"2026-02-22T23:16:45.419958+0000","last_undegraded":"2026-02-22T23:16:45.419958+0000","last_fullsized":"2026-02-22T23:16:45.419958+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T23:03:00.647409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T23:03:00.647409+0000","last_clean_scrub_stamp":"2026-02-22T23:03:00.647409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T08:32:13.395954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00046171899999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":1337,"reported_epoch":671,"state":"active+clean","last_fresh":"2026-02-22T23:16:45.420081+0000","last_change":"2026-02-22T23:03:03.657941+0000","last_active":"2026-02-22T23:16:45.420081+0000","last_peered":"2026-02-22T23:16:45.420081+0000","last_clean":"2026-02-22T23:16:45.420081+0000","last_became_active":"2026-02-22T23:03:01.654932+0000","last_became_peered":"2026-02-22T23:03:01.654932+0000","last_unstale":"2026-02-22T23:16:45.420081+0000","last_undegraded":"2026-02-22T23:16:45.420081+0000","last_fullsized":"2026-02-22T23:16:45.420081+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T23:03:00.647409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T23:03:00.647409+0000","last_clean_scrub_stamp":"2026-02-22T23:03:00.647409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T00:40:58.786037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00092843799999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":1337,"reported_epoch":671,"state":"active+clean","last_fresh":"2026-02-22T23:16:45.420260+0000","last_change":"2026-02-22T23:03:03.657748+0000","last_active":"2026-02-22T23:16:45.420260+0000","last_peered":"2026-02-22T23:16:45.420260+0000","last_clean":"2026-02-22T23:16:45.420260+0000","last_became_active":"2026-02-22T23:03:01.655313+0000","last_became_peered":"2026-02-22T23:03:01.655313+0000","last_unstale":"2026-02-22T23:16:45.420260+0000","last_undegraded":"2026-02-22T23:16:45.420260+0000","last_fullsized":"2026-02-22T23:16:45.420260+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T23:03:00.647409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T23:03:00.647409+0000","last_clean_scrub_stamp":"2026-02-22T23:03:00.647409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T06:55:02.798991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077562300000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359738538,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32344,"kb_used_data":5380,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94339496,"statfs":{"total":96636764160,"available":96603643904,"internally_reserved":0,"allocated":5509120,"data_stored":2598538,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27589062},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":8,"seq":34359738537,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":81520,"kb_used_data":5976,"kb_used_omap":1,"kb_used_meta":75518,"kb_avail":94290320,"statfs":{"total":96636764160,"available":96553287680,"internally_reserved":0,"allocated":6119424,"data_stored":3188965,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":77330886},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738537,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":63464,"kb_used_data":5968,"kb_used_omap":1,"kb_used_meta":57470,"kb_avail":94308376,"statfs":{"total":96636764160,"available":96571777024,"internally_reserved":0,"allocated":6111232,"data_stored":3188911,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58849734},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T23:16:50.542 DEBUG:teuthology.orchestra.run.trial168:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T23:16:50.702 INFO:teuthology.orchestra.run.trial168.stdout: 2026-02-22T23:16:50.703 INFO:teuthology.orchestra.run.trial168.stderr:dumped all 2026-02-22T23:16:50.710 INFO:teuthology.orchestra.run.trial168.stdout:{"pg_ready":true,"pg_map":{"version":1089,"stamp":"2026-02-22T23:16:49.348195+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":103,"num_write_kb":1630,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":75,"ondisk_log_size":75,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":177328,"kb_used_data":17324,"kb_used_omap":4,"kb_used_meta":159931,"kb_avail":282938192,"statfs":{"total":289910292480,"available":289728708608,"internally_reserved":0,"allocated":17739776,"data_stored":8976414,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4782,"internal_metadata":163769682},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-43,"num_objects":-9,"num_object_clones":0,"num_object_copies":-18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-9,"num_whiteouts":0,"num_read":-365982,"num_read_kb":-295194,"num_write":-7769,"num_write_kb":-78746,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.616678"},"pg_stats":[{"pgid":"1.0","version":"10'72","reported_seq":1428,"reported_epoch":671,"state":"active+clean","last_fresh":"2026-02-22T23:16:45.419414+0000","last_change":"2026-02-22T23:02:56.646381+0000","last_active":"2026-02-22T23:16:45.419414+0000","last_peered":"2026-02-22T23:16:45.419414+0000","last_clean":"2026-02-22T23:16:45.419414+0000","last_became_active":"2026-02-22T23:02:56.645752+0000","last_became_peered":"2026-02-22T23:02:56.645752+0000","last_unstale":"2026-02-22T23:16:45.419414+0000","last_undegraded":"2026-02-22T23:16:45.419414+0000","last_fullsized":"2026-02-22T23:16:45.419414+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T23:02:55.640202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T23:02:55.640202+0000","last_clean_scrub_stamp":"2026-02-22T23:02:55.640202+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T05:06:17.108040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"13'1","reported_seq":1338,"reported_epoch":671,"state":"active+clean","last_fresh":"2026-02-22T23:16:45.419942+0000","last_change":"2026-02-22T23:03:03.657709+0000","last_active":"2026-02-22T23:16:45.419942+0000","last_peered":"2026-02-22T23:16:45.419942+0000","last_clean":"2026-02-22T23:16:45.419942+0000","last_became_active":"2026-02-22T23:03:01.654744+0000","last_became_peered":"2026-02-22T23:03:01.654744+0000","last_unstale":"2026-02-22T23:16:45.419942+0000","last_undegraded":"2026-02-22T23:16:45.419942+0000","last_fullsized":"2026-02-22T23:16:45.419942+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T23:03:00.647409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T23:03:00.647409+0000","last_clean_scrub_stamp":"2026-02-22T23:03:00.647409+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T08:30:18.919810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00079898700000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":1297,"reported_epoch":671,"state":"active+clean","last_fresh":"2026-02-22T23:16:45.422453+0000","last_change":"2026-02-22T23:03:04.197929+0000","last_active":"2026-02-22T23:16:45.422453+0000","last_peered":"2026-02-22T23:16:45.422453+0000","last_clean":"2026-02-22T23:16:45.422453+0000","last_became_active":"2026-02-22T23:03:01.655068+0000","last_became_peered":"2026-02-22T23:03:01.655068+0000","last_unstale":"2026-02-22T23:16:45.422453+0000","last_undegraded":"2026-02-22T23:16:45.422453+0000","last_fullsized":"2026-02-22T23:16:45.422453+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T23:03:00.647409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T23:03:00.647409+0000","last_clean_scrub_stamp":"2026-02-22T23:03:00.647409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T00:00:11.473581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00033282899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":1297,"reported_epoch":671,"state":"active+clean","last_fresh":"2026-02-22T23:16:45.422218+0000","last_change":"2026-02-22T23:03:04.197991+0000","last_active":"2026-02-22T23:16:45.422218+0000","last_peered":"2026-02-22T23:16:45.422218+0000","last_clean":"2026-02-22T23:16:45.422218+0000","last_became_active":"2026-02-22T23:03:01.655046+0000","last_became_peered":"2026-02-22T23:03:01.655046+0000","last_unstale":"2026-02-22T23:16:45.422218+0000","last_undegraded":"2026-02-22T23:16:45.422218+0000","last_fullsized":"2026-02-22T23:16:45.422218+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T23:03:00.647409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T23:03:00.647409+0000","last_clean_scrub_stamp":"2026-02-22T23:03:00.647409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T05:07:31.304712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00038590800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"15'2","reported_seq":1337,"reported_epoch":671,"state":"active+clean","last_fresh":"2026-02-22T23:16:45.426754+0000","last_change":"2026-02-22T23:03:03.656168+0000","last_active":"2026-02-22T23:16:45.426754+0000","last_peered":"2026-02-22T23:16:45.426754+0000","last_clean":"2026-02-22T23:16:45.426754+0000","last_became_active":"2026-02-22T23:03:01.655020+0000","last_became_peered":"2026-02-22T23:03:01.655020+0000","last_unstale":"2026-02-22T23:16:45.426754+0000","last_undegraded":"2026-02-22T23:16:45.426754+0000","last_fullsized":"2026-02-22T23:16:45.426754+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T23:03:00.647409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T23:03:00.647409+0000","last_clean_scrub_stamp":"2026-02-22T23:03:00.647409+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T01:25:19.891118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000724969,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":1337,"reported_epoch":671,"state":"active+clean","last_fresh":"2026-02-22T23:16:45.420161+0000","last_change":"2026-02-22T23:03:03.657389+0000","last_active":"2026-02-22T23:16:45.420161+0000","last_peered":"2026-02-22T23:16:45.420161+0000","last_clean":"2026-02-22T23:16:45.420161+0000","last_became_active":"2026-02-22T23:03:01.654989+0000","last_became_peered":"2026-02-22T23:03:01.654989+0000","last_unstale":"2026-02-22T23:16:45.420161+0000","last_undegraded":"2026-02-22T23:16:45.420161+0000","last_fullsized":"2026-02-22T23:16:45.420161+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T23:03:00.647409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T23:03:00.647409+0000","last_clean_scrub_stamp":"2026-02-22T23:03:00.647409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T01:06:44.014188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000472108,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":1337,"reported_epoch":671,"state":"active+clean","last_fresh":"2026-02-22T23:16:45.419958+0000","last_change":"2026-02-22T23:03:03.657408+0000","last_active":"2026-02-22T23:16:45.419958+0000","last_peered":"2026-02-22T23:16:45.419958+0000","last_clean":"2026-02-22T23:16:45.419958+0000","last_became_active":"2026-02-22T23:03:01.655252+0000","last_became_peered":"2026-02-22T23:03:01.655252+0000","last_unstale":"2026-02-22T23:16:45.419958+0000","last_undegraded":"2026-02-22T23:16:45.419958+0000","last_fullsized":"2026-02-22T23:16:45.419958+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T23:03:00.647409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T23:03:00.647409+0000","last_clean_scrub_stamp":"2026-02-22T23:03:00.647409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T08:32:13.395954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00046171899999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":1337,"reported_epoch":671,"state":"active+clean","last_fresh":"2026-02-22T23:16:45.420081+0000","last_change":"2026-02-22T23:03:03.657941+0000","last_active":"2026-02-22T23:16:45.420081+0000","last_peered":"2026-02-22T23:16:45.420081+0000","last_clean":"2026-02-22T23:16:45.420081+0000","last_became_active":"2026-02-22T23:03:01.654932+0000","last_became_peered":"2026-02-22T23:03:01.654932+0000","last_unstale":"2026-02-22T23:16:45.420081+0000","last_undegraded":"2026-02-22T23:16:45.420081+0000","last_fullsized":"2026-02-22T23:16:45.420081+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T23:03:00.647409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T23:03:00.647409+0000","last_clean_scrub_stamp":"2026-02-22T23:03:00.647409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T00:40:58.786037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00092843799999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":1337,"reported_epoch":671,"state":"active+clean","last_fresh":"2026-02-22T23:16:45.420260+0000","last_change":"2026-02-22T23:03:03.657748+0000","last_active":"2026-02-22T23:16:45.420260+0000","last_peered":"2026-02-22T23:16:45.420260+0000","last_clean":"2026-02-22T23:16:45.420260+0000","last_became_active":"2026-02-22T23:03:01.655313+0000","last_became_peered":"2026-02-22T23:03:01.655313+0000","last_unstale":"2026-02-22T23:16:45.420260+0000","last_undegraded":"2026-02-22T23:16:45.420260+0000","last_fullsized":"2026-02-22T23:16:45.420260+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T23:03:00.647409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T23:03:00.647409+0000","last_clean_scrub_stamp":"2026-02-22T23:03:00.647409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T06:55:02.798991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077562300000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359738538,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32344,"kb_used_data":5380,"kb_used_omap":1,"kb_used_meta":26942,"kb_avail":94339496,"statfs":{"total":96636764160,"available":96603643904,"internally_reserved":0,"allocated":5509120,"data_stored":2598538,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27589062},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":8,"seq":34359738537,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":81520,"kb_used_data":5976,"kb_used_omap":1,"kb_used_meta":75518,"kb_avail":94290320,"statfs":{"total":96636764160,"available":96553287680,"internally_reserved":0,"allocated":6119424,"data_stored":3188965,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":77330886},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738537,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":63464,"kb_used_data":5968,"kb_used_omap":1,"kb_used_meta":57470,"kb_avail":94308376,"statfs":{"total":96636764160,"available":96571777024,"internally_reserved":0,"allocated":6111232,"data_stored":3188911,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58849734},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T23:16:50.712 INFO:tasks.ceph.ceph_manager.ceph:clean! 2026-02-22T23:16:50.712 DEBUG:teuthology.orchestra.run.trial168:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T23:16:50.872 INFO:teuthology.orchestra.run.trial168.stdout: 2026-02-22T23:16:50.873 INFO:teuthology.orchestra.run.trial168.stderr:dumped all 2026-02-22T23:16:50.881 INFO:teuthology.orchestra.run.trial168.stdout:{"pg_ready":true,"pg_map":{"version":1090,"stamp":"2026-02-22T23:16:50.732245+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":103,"num_write_kb":1630,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":75,"ondisk_log_size":75,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":169060,"kb_used_data":17380,"kb_used_omap":4,"kb_used_meta":151611,"kb_avail":282946460,"statfs":{"total":289910292480,"available":289737175040,"internally_reserved":0,"allocated":17797120,"data_stored":9015556,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4782,"internal_metadata":155250002},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-43,"num_objects":-9,"num_object_clones":0,"num_object_copies":-18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-9,"num_whiteouts":0,"num_read":-365982,"num_read_kb":-295194,"num_write":-7769,"num_write_kb":-78746,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.314256"},"pg_stats":[{"pgid":"1.0","version":"10'72","reported_seq":1434,"reported_epoch":674,"state":"active+clean","last_fresh":"2026-02-22T23:16:50.525176+0000","last_change":"2026-02-22T23:02:56.646381+0000","last_active":"2026-02-22T23:16:50.525176+0000","last_peered":"2026-02-22T23:16:50.525176+0000","last_clean":"2026-02-22T23:16:50.525176+0000","last_became_active":"2026-02-22T23:02:56.645752+0000","last_became_peered":"2026-02-22T23:02:56.645752+0000","last_unstale":"2026-02-22T23:16:50.525176+0000","last_undegraded":"2026-02-22T23:16:50.525176+0000","last_fullsized":"2026-02-22T23:16:50.525176+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T23:02:55.640202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T23:02:55.640202+0000","last_clean_scrub_stamp":"2026-02-22T23:02:55.640202+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T05:06:17.108040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"13'1","reported_seq":1344,"reported_epoch":674,"state":"active+clean","last_fresh":"2026-02-22T23:16:50.525388+0000","last_change":"2026-02-22T23:03:03.657709+0000","last_active":"2026-02-22T23:16:50.525388+0000","last_peered":"2026-02-22T23:16:50.525388+0000","last_clean":"2026-02-22T23:16:50.525388+0000","last_became_active":"2026-02-22T23:03:01.654744+0000","last_became_peered":"2026-02-22T23:03:01.654744+0000","last_unstale":"2026-02-22T23:16:50.525388+0000","last_undegraded":"2026-02-22T23:16:50.525388+0000","last_fullsized":"2026-02-22T23:16:50.525388+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T23:03:00.647409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T23:03:00.647409+0000","last_clean_scrub_stamp":"2026-02-22T23:03:00.647409+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T08:30:18.919810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00079898700000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":1301,"reported_epoch":673,"state":"active+clean","last_fresh":"2026-02-22T23:16:47.348568+0000","last_change":"2026-02-22T23:03:04.197929+0000","last_active":"2026-02-22T23:16:47.348568+0000","last_peered":"2026-02-22T23:16:47.348568+0000","last_clean":"2026-02-22T23:16:47.348568+0000","last_became_active":"2026-02-22T23:03:01.655068+0000","last_became_peered":"2026-02-22T23:03:01.655068+0000","last_unstale":"2026-02-22T23:16:47.348568+0000","last_undegraded":"2026-02-22T23:16:47.348568+0000","last_fullsized":"2026-02-22T23:16:47.348568+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T23:03:00.647409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T23:03:00.647409+0000","last_clean_scrub_stamp":"2026-02-22T23:03:00.647409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T00:00:11.473581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00033282899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":1301,"reported_epoch":673,"state":"active+clean","last_fresh":"2026-02-22T23:16:47.348766+0000","last_change":"2026-02-22T23:03:04.197991+0000","last_active":"2026-02-22T23:16:47.348766+0000","last_peered":"2026-02-22T23:16:47.348766+0000","last_clean":"2026-02-22T23:16:47.348766+0000","last_became_active":"2026-02-22T23:03:01.655046+0000","last_became_peered":"2026-02-22T23:03:01.655046+0000","last_unstale":"2026-02-22T23:16:47.348766+0000","last_undegraded":"2026-02-22T23:16:47.348766+0000","last_fullsized":"2026-02-22T23:16:47.348766+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T23:03:00.647409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T23:03:00.647409+0000","last_clean_scrub_stamp":"2026-02-22T23:03:00.647409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T05:07:31.304712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00038590800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"15'2","reported_seq":1343,"reported_epoch":674,"state":"active+clean","last_fresh":"2026-02-22T23:16:49.350597+0000","last_change":"2026-02-22T23:03:03.656168+0000","last_active":"2026-02-22T23:16:49.350597+0000","last_peered":"2026-02-22T23:16:49.350597+0000","last_clean":"2026-02-22T23:16:49.350597+0000","last_became_active":"2026-02-22T23:03:01.655020+0000","last_became_peered":"2026-02-22T23:03:01.655020+0000","last_unstale":"2026-02-22T23:16:49.350597+0000","last_undegraded":"2026-02-22T23:16:49.350597+0000","last_fullsized":"2026-02-22T23:16:49.350597+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T23:03:00.647409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T23:03:00.647409+0000","last_clean_scrub_stamp":"2026-02-22T23:03:00.647409+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T01:25:19.891118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000724969,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":1343,"reported_epoch":674,"state":"active+clean","last_fresh":"2026-02-22T23:16:50.526261+0000","last_change":"2026-02-22T23:03:03.657389+0000","last_active":"2026-02-22T23:16:50.526261+0000","last_peered":"2026-02-22T23:16:50.526261+0000","last_clean":"2026-02-22T23:16:50.526261+0000","last_became_active":"2026-02-22T23:03:01.654989+0000","last_became_peered":"2026-02-22T23:03:01.654989+0000","last_unstale":"2026-02-22T23:16:50.526261+0000","last_undegraded":"2026-02-22T23:16:50.526261+0000","last_fullsized":"2026-02-22T23:16:50.526261+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T23:03:00.647409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T23:03:00.647409+0000","last_clean_scrub_stamp":"2026-02-22T23:03:00.647409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T01:06:44.014188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000472108,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":1343,"reported_epoch":674,"state":"active+clean","last_fresh":"2026-02-22T23:16:50.525532+0000","last_change":"2026-02-22T23:03:03.657408+0000","last_active":"2026-02-22T23:16:50.525532+0000","last_peered":"2026-02-22T23:16:50.525532+0000","last_clean":"2026-02-22T23:16:50.525532+0000","last_became_active":"2026-02-22T23:03:01.655252+0000","last_became_peered":"2026-02-22T23:03:01.655252+0000","last_unstale":"2026-02-22T23:16:50.525532+0000","last_undegraded":"2026-02-22T23:16:50.525532+0000","last_fullsized":"2026-02-22T23:16:50.525532+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T23:03:00.647409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T23:03:00.647409+0000","last_clean_scrub_stamp":"2026-02-22T23:03:00.647409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T08:32:13.395954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00046171899999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":1343,"reported_epoch":674,"state":"active+clean","last_fresh":"2026-02-22T23:16:50.525652+0000","last_change":"2026-02-22T23:03:03.657941+0000","last_active":"2026-02-22T23:16:50.525652+0000","last_peered":"2026-02-22T23:16:50.525652+0000","last_clean":"2026-02-22T23:16:50.525652+0000","last_became_active":"2026-02-22T23:03:01.654932+0000","last_became_peered":"2026-02-22T23:03:01.654932+0000","last_unstale":"2026-02-22T23:16:50.525652+0000","last_undegraded":"2026-02-22T23:16:50.525652+0000","last_fullsized":"2026-02-22T23:16:50.525652+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T23:03:00.647409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T23:03:00.647409+0000","last_clean_scrub_stamp":"2026-02-22T23:03:00.647409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T00:40:58.786037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00092843799999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":1343,"reported_epoch":674,"state":"active+clean","last_fresh":"2026-02-22T23:16:50.526192+0000","last_change":"2026-02-22T23:03:03.657748+0000","last_active":"2026-02-22T23:16:50.526192+0000","last_peered":"2026-02-22T23:16:50.526192+0000","last_clean":"2026-02-22T23:16:50.526192+0000","last_became_active":"2026-02-22T23:03:01.655313+0000","last_became_peered":"2026-02-22T23:03:01.655313+0000","last_unstale":"2026-02-22T23:16:50.526192+0000","last_undegraded":"2026-02-22T23:16:50.526192+0000","last_fullsized":"2026-02-22T23:16:50.526192+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T23:03:00.647409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T23:03:00.647409+0000","last_clean_scrub_stamp":"2026-02-22T23:03:00.647409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T06:55:02.798991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077562300000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359738539,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36524,"kb_used_data":5400,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94335316,"statfs":{"total":96636764160,"available":96599363584,"internally_reserved":0,"allocated":5529600,"data_stored":2610404,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":8,"seq":34359738538,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":69052,"kb_used_data":5988,"kb_used_omap":1,"kb_used_meta":63038,"kb_avail":94302788,"statfs":{"total":96636764160,"available":96566054912,"internally_reserved":0,"allocated":6131712,"data_stored":3200799,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":64551366},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738538,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":63484,"kb_used_data":5992,"kb_used_omap":1,"kb_used_meta":57470,"kb_avail":94308356,"statfs":{"total":96636764160,"available":96571756544,"internally_reserved":0,"allocated":6135808,"data_stored":3204353,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58849734},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T23:16:50.882 DEBUG:teuthology.orchestra.run.trial168:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2026-02-22T23:16:51.042 INFO:teuthology.orchestra.run.trial168.stdout: 2026-02-22T23:16:51.043 INFO:teuthology.orchestra.run.trial168.stdout:{"epoch":674,"fsid":"7798f833-50d5-4df5-a39c-9dab1506b7db","created":"2026-02-22T23:02:46.276170+0000","modified":"2026-02-22T23:16:49.347144+0000","last_up_change":"2026-02-22T23:02:54.636216+0000","last_in_change":"2026-02-22T23:02:46.837523+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2026-02-22T23:02:54.645801+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"11","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.9900000095367432,"score_stable":2.9900000095367432,"optimal_score":0.67000001668930054,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2026-02-22T23:03:00.444070+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":15,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"882c47f4-e016-46e9-b67e-b2d6389fc9f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":577,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.168:6808","nonce":3554969634},{"type":"v1","addr":"10.20.193.168:6809","nonce":3554969634}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.168:6810","nonce":3554969634},{"type":"v1","addr":"10.20.193.168:6811","nonce":3554969634}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.168:6814","nonce":3554969634},{"type":"v1","addr":"10.20.193.168:6815","nonce":3554969634}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.168:6812","nonce":3554969634},{"type":"v1","addr":"10.20.193.168:6813","nonce":3554969634}]},"public_addr":"10.20.193.168:6809/3554969634","cluster_addr":"10.20.193.168:6811/3554969634","heartbeat_back_addr":"10.20.193.168:6815/3554969634","heartbeat_front_addr":"10.20.193.168:6813/3554969634","state":["exists","up"]},{"osd":1,"uuid":"9835b2d4-f84f-4f74-b135-663786caa176","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":577,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.168:6816","nonce":1754802247},{"type":"v1","addr":"10.20.193.168:6817","nonce":1754802247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.168:6818","nonce":1754802247},{"type":"v1","addr":"10.20.193.168:6819","nonce":1754802247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.168:6822","nonce":1754802247},{"type":"v1","addr":"10.20.193.168:6823","nonce":1754802247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.168:6820","nonce":1754802247},{"type":"v1","addr":"10.20.193.168:6821","nonce":1754802247}]},"public_addr":"10.20.193.168:6817/1754802247","cluster_addr":"10.20.193.168:6819/1754802247","heartbeat_back_addr":"10.20.193.168:6823/1754802247","heartbeat_front_addr":"10.20.193.168:6821/1754802247","state":["exists","up"]},{"osd":2,"uuid":"59a03d64-5286-4376-b7d5-80007120931a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":577,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.168:6800","nonce":1816324481},{"type":"v1","addr":"10.20.193.168:6801","nonce":1816324481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.168:6802","nonce":1816324481},{"type":"v1","addr":"10.20.193.168:6803","nonce":1816324481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.168:6806","nonce":1816324481},{"type":"v1","addr":"10.20.193.168:6807","nonce":1816324481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.168:6804","nonce":1816324481},{"type":"v1","addr":"10.20.193.168:6805","nonce":1816324481}]},"public_addr":"10.20.193.168:6801/1816324481","cluster_addr":"10.20.193.168:6803/1816324481","heartbeat_back_addr":"10.20.193.168:6807/1816324481","heartbeat_front_addr":"10.20.193.168:6805/1816324481","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2026-02-22T23:02:53.012632+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2026-02-22T23:02:53.333411+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2026-02-22T23:02:53.320621+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":3,"snaps":[{"begin":323,"length":3}]},{"pool":4,"snaps":[{"begin":2,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2026-02-22T23:16:52.052 INFO:tasks.ceph:Scrubbing osd.0 2026-02-22T23:16:52.052 DEBUG:teuthology.orchestra.run.trial168:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2026-02-22T23:16:52.132 INFO:teuthology.orchestra.run.trial168.stdout:{ 2026-02-22T23:16:52.132 INFO:teuthology.orchestra.run.trial168.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2026-02-22T23:16:52.132 INFO:teuthology.orchestra.run.trial168.stdout:} 2026-02-22T23:16:52.137 DEBUG:teuthology.orchestra.run.trial168:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2026-02-22T23:16:52.296 INFO:teuthology.orchestra.run.trial168.stderr:instructed osd(s) 0 to deep-scrub 2026-02-22T23:16:52.304 INFO:tasks.ceph:Scrubbing osd.1 2026-02-22T23:16:52.304 DEBUG:teuthology.orchestra.run.trial168:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2026-02-22T23:16:52.382 INFO:teuthology.orchestra.run.trial168.stdout:{ 2026-02-22T23:16:52.382 INFO:teuthology.orchestra.run.trial168.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2026-02-22T23:16:52.382 INFO:teuthology.orchestra.run.trial168.stdout:} 2026-02-22T23:16:52.387 DEBUG:teuthology.orchestra.run.trial168:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2026-02-22T23:16:52.543 INFO:teuthology.orchestra.run.trial168.stderr:instructed osd(s) 1 to deep-scrub 2026-02-22T23:16:52.551 INFO:tasks.ceph:Scrubbing osd.2 2026-02-22T23:16:52.551 DEBUG:teuthology.orchestra.run.trial168:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2026-02-22T23:16:52.630 INFO:teuthology.orchestra.run.trial168.stdout:{ 2026-02-22T23:16:52.630 INFO:teuthology.orchestra.run.trial168.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2026-02-22T23:16:52.630 INFO:teuthology.orchestra.run.trial168.stdout:} 2026-02-22T23:16:52.636 DEBUG:teuthology.orchestra.run.trial168:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2026-02-22T23:16:52.791 INFO:teuthology.orchestra.run.trial168.stderr:instructed osd(s) 2 to deep-scrub 2026-02-22T23:16:52.799 DEBUG:teuthology.orchestra.run.trial168:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T23:16:52.958 INFO:teuthology.orchestra.run.trial168.stdout: 2026-02-22T23:16:52.958 INFO:teuthology.orchestra.run.trial168.stderr:dumped all 2026-02-22T23:16:52.966 INFO:teuthology.orchestra.run.trial168.stdout:{"pg_ready":true,"pg_map":{"version":1091,"stamp":"2026-02-22T23:16:52.732457+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":103,"num_write_kb":1630,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":75,"ondisk_log_size":75,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":169060,"kb_used_data":17380,"kb_used_omap":4,"kb_used_meta":151611,"kb_avail":282946460,"statfs":{"total":289910292480,"available":289737175040,"internally_reserved":0,"allocated":17797120,"data_stored":9015556,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4782,"internal_metadata":155250002},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-129,"num_objects":-13,"num_object_clones":0,"num_object_copies":-26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-13,"num_whiteouts":0,"num_read":-367637,"num_read_kb":-296612,"num_write":-7879,"num_write_kb":-78811,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.385830"},"pg_stats":[{"pgid":"1.0","version":"10'72","reported_seq":1434,"reported_epoch":674,"state":"active+clean","last_fresh":"2026-02-22T23:16:50.525176+0000","last_change":"2026-02-22T23:02:56.646381+0000","last_active":"2026-02-22T23:16:50.525176+0000","last_peered":"2026-02-22T23:16:50.525176+0000","last_clean":"2026-02-22T23:16:50.525176+0000","last_became_active":"2026-02-22T23:02:56.645752+0000","last_became_peered":"2026-02-22T23:02:56.645752+0000","last_unstale":"2026-02-22T23:16:50.525176+0000","last_undegraded":"2026-02-22T23:16:50.525176+0000","last_fullsized":"2026-02-22T23:16:50.525176+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T23:02:55.640202+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T23:02:55.640202+0000","last_clean_scrub_stamp":"2026-02-22T23:02:55.640202+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T05:06:17.108040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"13'1","reported_seq":1344,"reported_epoch":674,"state":"active+clean","last_fresh":"2026-02-22T23:16:50.525388+0000","last_change":"2026-02-22T23:03:03.657709+0000","last_active":"2026-02-22T23:16:50.525388+0000","last_peered":"2026-02-22T23:16:50.525388+0000","last_clean":"2026-02-22T23:16:50.525388+0000","last_became_active":"2026-02-22T23:03:01.654744+0000","last_became_peered":"2026-02-22T23:03:01.654744+0000","last_unstale":"2026-02-22T23:16:50.525388+0000","last_undegraded":"2026-02-22T23:16:50.525388+0000","last_fullsized":"2026-02-22T23:16:50.525388+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T23:03:00.647409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T23:03:00.647409+0000","last_clean_scrub_stamp":"2026-02-22T23:03:00.647409+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T08:30:18.919810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00079898700000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":1301,"reported_epoch":673,"state":"active+clean","last_fresh":"2026-02-22T23:16:47.348568+0000","last_change":"2026-02-22T23:03:04.197929+0000","last_active":"2026-02-22T23:16:47.348568+0000","last_peered":"2026-02-22T23:16:47.348568+0000","last_clean":"2026-02-22T23:16:47.348568+0000","last_became_active":"2026-02-22T23:03:01.655068+0000","last_became_peered":"2026-02-22T23:03:01.655068+0000","last_unstale":"2026-02-22T23:16:47.348568+0000","last_undegraded":"2026-02-22T23:16:47.348568+0000","last_fullsized":"2026-02-22T23:16:47.348568+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T23:03:00.647409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T23:03:00.647409+0000","last_clean_scrub_stamp":"2026-02-22T23:03:00.647409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T00:00:11.473581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00033282899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":1301,"reported_epoch":673,"state":"active+clean","last_fresh":"2026-02-22T23:16:47.348766+0000","last_change":"2026-02-22T23:03:04.197991+0000","last_active":"2026-02-22T23:16:47.348766+0000","last_peered":"2026-02-22T23:16:47.348766+0000","last_clean":"2026-02-22T23:16:47.348766+0000","last_became_active":"2026-02-22T23:03:01.655046+0000","last_became_peered":"2026-02-22T23:03:01.655046+0000","last_unstale":"2026-02-22T23:16:47.348766+0000","last_undegraded":"2026-02-22T23:16:47.348766+0000","last_fullsized":"2026-02-22T23:16:47.348766+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T23:03:00.647409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T23:03:00.647409+0000","last_clean_scrub_stamp":"2026-02-22T23:03:00.647409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T05:07:31.304712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00038590800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"15'2","reported_seq":1343,"reported_epoch":674,"state":"active+clean","last_fresh":"2026-02-22T23:16:49.350597+0000","last_change":"2026-02-22T23:03:03.656168+0000","last_active":"2026-02-22T23:16:49.350597+0000","last_peered":"2026-02-22T23:16:49.350597+0000","last_clean":"2026-02-22T23:16:49.350597+0000","last_became_active":"2026-02-22T23:03:01.655020+0000","last_became_peered":"2026-02-22T23:03:01.655020+0000","last_unstale":"2026-02-22T23:16:49.350597+0000","last_undegraded":"2026-02-22T23:16:49.350597+0000","last_fullsized":"2026-02-22T23:16:49.350597+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T23:03:00.647409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T23:03:00.647409+0000","last_clean_scrub_stamp":"2026-02-22T23:03:00.647409+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T01:25:19.891118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000724969,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":1343,"reported_epoch":674,"state":"active+clean","last_fresh":"2026-02-22T23:16:50.526261+0000","last_change":"2026-02-22T23:03:03.657389+0000","last_active":"2026-02-22T23:16:50.526261+0000","last_peered":"2026-02-22T23:16:50.526261+0000","last_clean":"2026-02-22T23:16:50.526261+0000","last_became_active":"2026-02-22T23:03:01.654989+0000","last_became_peered":"2026-02-22T23:03:01.654989+0000","last_unstale":"2026-02-22T23:16:50.526261+0000","last_undegraded":"2026-02-22T23:16:50.526261+0000","last_fullsized":"2026-02-22T23:16:50.526261+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T23:03:00.647409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T23:03:00.647409+0000","last_clean_scrub_stamp":"2026-02-22T23:03:00.647409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T01:06:44.014188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000472108,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":1343,"reported_epoch":674,"state":"active+clean","last_fresh":"2026-02-22T23:16:50.525532+0000","last_change":"2026-02-22T23:03:03.657408+0000","last_active":"2026-02-22T23:16:50.525532+0000","last_peered":"2026-02-22T23:16:50.525532+0000","last_clean":"2026-02-22T23:16:50.525532+0000","last_became_active":"2026-02-22T23:03:01.655252+0000","last_became_peered":"2026-02-22T23:03:01.655252+0000","last_unstale":"2026-02-22T23:16:50.525532+0000","last_undegraded":"2026-02-22T23:16:50.525532+0000","last_fullsized":"2026-02-22T23:16:50.525532+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T23:03:00.647409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T23:03:00.647409+0000","last_clean_scrub_stamp":"2026-02-22T23:03:00.647409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T08:32:13.395954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00046171899999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":1343,"reported_epoch":674,"state":"active+clean","last_fresh":"2026-02-22T23:16:50.525652+0000","last_change":"2026-02-22T23:03:03.657941+0000","last_active":"2026-02-22T23:16:50.525652+0000","last_peered":"2026-02-22T23:16:50.525652+0000","last_clean":"2026-02-22T23:16:50.525652+0000","last_became_active":"2026-02-22T23:03:01.654932+0000","last_became_peered":"2026-02-22T23:03:01.654932+0000","last_unstale":"2026-02-22T23:16:50.525652+0000","last_undegraded":"2026-02-22T23:16:50.525652+0000","last_fullsized":"2026-02-22T23:16:50.525652+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T23:03:00.647409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T23:03:00.647409+0000","last_clean_scrub_stamp":"2026-02-22T23:03:00.647409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T00:40:58.786037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00092843799999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":1343,"reported_epoch":674,"state":"active+clean","last_fresh":"2026-02-22T23:16:50.526192+0000","last_change":"2026-02-22T23:03:03.657748+0000","last_active":"2026-02-22T23:16:50.526192+0000","last_peered":"2026-02-22T23:16:50.526192+0000","last_clean":"2026-02-22T23:16:50.526192+0000","last_became_active":"2026-02-22T23:03:01.655313+0000","last_became_peered":"2026-02-22T23:03:01.655313+0000","last_unstale":"2026-02-22T23:16:50.526192+0000","last_undegraded":"2026-02-22T23:16:50.526192+0000","last_fullsized":"2026-02-22T23:16:50.526192+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T23:03:00.647409+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T23:03:00.647409+0000","last_clean_scrub_stamp":"2026-02-22T23:03:00.647409+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T06:55:02.798991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077562300000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359738539,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36524,"kb_used_data":5400,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94335316,"statfs":{"total":96636764160,"available":96599363584,"internally_reserved":0,"allocated":5529600,"data_stored":2610404,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":8,"seq":34359738538,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":69052,"kb_used_data":5988,"kb_used_omap":1,"kb_used_meta":63038,"kb_avail":94302788,"statfs":{"total":96636764160,"available":96566054912,"internally_reserved":0,"allocated":6131712,"data_stored":3200799,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":64551366},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738538,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":63484,"kb_used_data":5992,"kb_used_omap":1,"kb_used_meta":57470,"kb_avail":94308356,"statfs":{"total":96636764160,"available":96571756544,"internally_reserved":0,"allocated":6135808,"data_stored":3204353,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58849734},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T23:16:52.973 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2026-02-22T23:02:55.640202+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=22, tm_hour=23, tm_min=2, tm_sec=55, tm_wday=6, tm_yday=53, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=22, tm_hour=23, tm_min=16, tm_sec=51, tm_wday=6, tm_yday=53, tm_isdst=0) 2026-02-22T23:16:52.973 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2026-02-22T23:03:00.647409+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=22, tm_hour=23, tm_min=3, tm_sec=0, tm_wday=6, tm_yday=53, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=22, tm_hour=23, tm_min=16, tm_sec=51, tm_wday=6, tm_yday=53, tm_isdst=0) 2026-02-22T23:16:52.973 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2026-02-22T23:03:00.647409+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=22, tm_hour=23, tm_min=3, tm_sec=0, tm_wday=6, tm_yday=53, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=22, tm_hour=23, tm_min=16, tm_sec=51, tm_wday=6, tm_yday=53, tm_isdst=0) 2026-02-22T23:16:52.973 INFO:tasks.ceph:pgid 2.1 last_scrub_stamp 2026-02-22T23:03:00.647409+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=22, tm_hour=23, tm_min=3, tm_sec=0, tm_wday=6, tm_yday=53, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=22, tm_hour=23, tm_min=16, tm_sec=51, tm_wday=6, tm_yday=53, tm_isdst=0) 2026-02-22T23:16:52.973 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2026-02-22T23:03:00.647409+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=22, tm_hour=23, tm_min=3, tm_sec=0, tm_wday=6, tm_yday=53, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=22, tm_hour=23, tm_min=16, tm_sec=51, tm_wday=6, tm_yday=53, tm_isdst=0) 2026-02-22T23:16:52.974 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2026-02-22T23:03:00.647409+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=22, tm_hour=23, tm_min=3, tm_sec=0, tm_wday=6, tm_yday=53, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=22, tm_hour=23, tm_min=16, tm_sec=51, tm_wday=6, tm_yday=53, tm_isdst=0) 2026-02-22T23:16:52.974 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2026-02-22T23:03:00.647409+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=22, tm_hour=23, tm_min=3, tm_sec=0, tm_wday=6, tm_yday=53, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=22, tm_hour=23, tm_min=16, tm_sec=51, tm_wday=6, tm_yday=53, tm_isdst=0) 2026-02-22T23:16:52.974 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2026-02-22T23:03:00.647409+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=22, tm_hour=23, tm_min=3, tm_sec=0, tm_wday=6, tm_yday=53, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=22, tm_hour=23, tm_min=16, tm_sec=51, tm_wday=6, tm_yday=53, tm_isdst=0) 2026-02-22T23:16:52.974 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2026-02-22T23:03:00.647409+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=22, tm_hour=23, tm_min=3, tm_sec=0, tm_wday=6, tm_yday=53, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=22, tm_hour=23, tm_min=16, tm_sec=51, tm_wday=6, tm_yday=53, tm_isdst=0) 2026-02-22T23:16:52.974 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2026-02-22T23:17:12.975 DEBUG:teuthology.orchestra.run.trial168:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-22T23:17:13.136 INFO:teuthology.orchestra.run.trial168.stdout: 2026-02-22T23:17:13.137 INFO:teuthology.orchestra.run.trial168.stderr:dumped all 2026-02-22T23:17:13.145 INFO:teuthology.orchestra.run.trial168.stdout:{"pg_ready":true,"pg_map":{"version":1101,"stamp":"2026-02-22T23:17:12.734334+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":103,"num_write_kb":1630,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":169056,"kb_used_data":17392,"kb_used_omap":4,"kb_used_meta":151611,"kb_avail":282946464,"statfs":{"total":289910292480,"available":289737179136,"internally_reserved":0,"allocated":17809408,"data_stored":9022672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4782,"internal_metadata":155250002},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001075"},"pg_stats":[{"pgid":"1.0","version":"674'74","reported_seq":1446,"reported_epoch":674,"state":"active+clean","last_fresh":"2026-02-22T23:16:53.133000+0000","last_change":"2026-02-22T23:16:53.133000+0000","last_active":"2026-02-22T23:16:53.133000+0000","last_peered":"2026-02-22T23:16:53.133000+0000","last_clean":"2026-02-22T23:16:53.133000+0000","last_became_active":"2026-02-22T23:02:56.645752+0000","last_became_peered":"2026-02-22T23:02:56.645752+0000","last_unstale":"2026-02-22T23:16:53.133000+0000","last_undegraded":"2026-02-22T23:16:53.133000+0000","last_fullsized":"2026-02-22T23:16:53.133000+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"674'74","last_scrub_stamp":"2026-02-22T23:16:53.132835+0000","last_deep_scrub":"674'74","last_deep_scrub_stamp":"2026-02-22T23:16:53.132835+0000","last_clean_scrub_stamp":"2026-02-22T23:16:53.132835+0000","objects_scrubbed":2,"log_size":74,"log_dups_size":0,"ondisk_log_size":74,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T08:23:21.680572+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"674'2","reported_seq":1354,"reported_epoch":674,"state":"active+clean","last_fresh":"2026-02-22T23:16:57.041228+0000","last_change":"2026-02-22T23:16:57.041239+0000","last_active":"2026-02-22T23:16:57.041228+0000","last_peered":"2026-02-22T23:16:57.041228+0000","last_clean":"2026-02-22T23:16:57.041228+0000","last_became_active":"2026-02-22T23:03:01.654744+0000","last_became_peered":"2026-02-22T23:03:01.654744+0000","last_unstale":"2026-02-22T23:16:57.041228+0000","last_undegraded":"2026-02-22T23:16:57.041228+0000","last_fullsized":"2026-02-22T23:16:57.041228+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"674'2","last_scrub_stamp":"2026-02-22T23:16:57.041191+0000","last_deep_scrub":"674'2","last_deep_scrub_stamp":"2026-02-22T23:16:57.041191+0000","last_clean_scrub_stamp":"2026-02-22T23:16:57.041191+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:23:32.331369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00079898700000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":1313,"reported_epoch":674,"state":"active+clean","last_fresh":"2026-02-22T23:16:54.144037+0000","last_change":"2026-02-22T23:16:54.144037+0000","last_active":"2026-02-22T23:16:54.144037+0000","last_peered":"2026-02-22T23:16:54.144037+0000","last_clean":"2026-02-22T23:16:54.144037+0000","last_became_active":"2026-02-22T23:03:01.655068+0000","last_became_peered":"2026-02-22T23:03:01.655068+0000","last_unstale":"2026-02-22T23:16:54.144037+0000","last_undegraded":"2026-02-22T23:16:54.144037+0000","last_fullsized":"2026-02-22T23:16:54.144037+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T23:16:54.143964+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T23:16:54.143964+0000","last_clean_scrub_stamp":"2026-02-22T23:16:54.143964+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T09:38:50.604078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00033282899999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":1312,"reported_epoch":674,"state":"active+clean","last_fresh":"2026-02-22T23:16:53.172588+0000","last_change":"2026-02-22T23:16:53.172602+0000","last_active":"2026-02-22T23:16:53.172588+0000","last_peered":"2026-02-22T23:16:53.172588+0000","last_clean":"2026-02-22T23:16:53.172588+0000","last_became_active":"2026-02-22T23:03:01.655046+0000","last_became_peered":"2026-02-22T23:03:01.655046+0000","last_unstale":"2026-02-22T23:16:53.172588+0000","last_undegraded":"2026-02-22T23:16:53.172588+0000","last_fullsized":"2026-02-22T23:16:53.172588+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T23:16:53.172540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T23:16:53.172540+0000","last_clean_scrub_stamp":"2026-02-22T23:16:53.172540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T05:11:05.133480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00038590800000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"674'3","reported_seq":1353,"reported_epoch":674,"state":"active+clean","last_fresh":"2026-02-22T23:16:52.712449+0000","last_change":"2026-02-22T23:16:52.712463+0000","last_active":"2026-02-22T23:16:52.712449+0000","last_peered":"2026-02-22T23:16:52.712449+0000","last_clean":"2026-02-22T23:16:52.712449+0000","last_became_active":"2026-02-22T23:03:01.655020+0000","last_became_peered":"2026-02-22T23:03:01.655020+0000","last_unstale":"2026-02-22T23:16:52.712449+0000","last_undegraded":"2026-02-22T23:16:52.712449+0000","last_fullsized":"2026-02-22T23:16:52.712449+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"674'3","last_scrub_stamp":"2026-02-22T23:16:52.712404+0000","last_deep_scrub":"674'3","last_deep_scrub_stamp":"2026-02-22T23:16:52.712404+0000","last_clean_scrub_stamp":"2026-02-22T23:16:52.712404+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T03:04:31.779924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000724969,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":1352,"reported_epoch":674,"state":"active+clean","last_fresh":"2026-02-22T23:16:55.098570+0000","last_change":"2026-02-22T23:16:55.098587+0000","last_active":"2026-02-22T23:16:55.098570+0000","last_peered":"2026-02-22T23:16:55.098570+0000","last_clean":"2026-02-22T23:16:55.098570+0000","last_became_active":"2026-02-22T23:03:01.654989+0000","last_became_peered":"2026-02-22T23:03:01.654989+0000","last_unstale":"2026-02-22T23:16:55.098570+0000","last_undegraded":"2026-02-22T23:16:55.098570+0000","last_fullsized":"2026-02-22T23:16:55.098570+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T23:16:55.098490+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T23:16:55.098490+0000","last_clean_scrub_stamp":"2026-02-22T23:16:55.098490+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T00:38:21.566677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000472108,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":1352,"reported_epoch":674,"state":"active+clean","last_fresh":"2026-02-22T23:16:56.089787+0000","last_change":"2026-02-22T23:16:56.089799+0000","last_active":"2026-02-22T23:16:56.089787+0000","last_peered":"2026-02-22T23:16:56.089787+0000","last_clean":"2026-02-22T23:16:56.089787+0000","last_became_active":"2026-02-22T23:03:01.655252+0000","last_became_peered":"2026-02-22T23:03:01.655252+0000","last_unstale":"2026-02-22T23:16:56.089787+0000","last_undegraded":"2026-02-22T23:16:56.089787+0000","last_fullsized":"2026-02-22T23:16:56.089787+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T23:16:56.089740+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T23:16:56.089740+0000","last_clean_scrub_stamp":"2026-02-22T23:16:56.089740+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T07:32:32.819126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00046171899999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":1352,"reported_epoch":674,"state":"active+clean","last_fresh":"2026-02-22T23:16:54.112987+0000","last_change":"2026-02-22T23:16:54.112997+0000","last_active":"2026-02-22T23:16:54.112987+0000","last_peered":"2026-02-22T23:16:54.112987+0000","last_clean":"2026-02-22T23:16:54.112987+0000","last_became_active":"2026-02-22T23:03:01.654932+0000","last_became_peered":"2026-02-22T23:03:01.654932+0000","last_unstale":"2026-02-22T23:16:54.112987+0000","last_undegraded":"2026-02-22T23:16:54.112987+0000","last_fullsized":"2026-02-22T23:16:54.112987+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T23:16:54.112952+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T23:16:54.112952+0000","last_clean_scrub_stamp":"2026-02-22T23:16:54.112952+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T10:15:50.883675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00092843799999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":1352,"reported_epoch":674,"state":"active+clean","last_fresh":"2026-02-22T23:16:58.070671+0000","last_change":"2026-02-22T23:16:58.070686+0000","last_active":"2026-02-22T23:16:58.070671+0000","last_peered":"2026-02-22T23:16:58.070671+0000","last_clean":"2026-02-22T23:16:58.070671+0000","last_became_active":"2026-02-22T23:03:01.655313+0000","last_became_peered":"2026-02-22T23:03:01.655313+0000","last_unstale":"2026-02-22T23:16:58.070671+0000","last_undegraded":"2026-02-22T23:16:58.070671+0000","last_fullsized":"2026-02-22T23:16:58.070671+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-22T23:16:58.070621+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-22T23:16:58.070621+0000","last_clean_scrub_stamp":"2026-02-22T23:16:58.070621+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-24T10:14:03.231066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00077562300000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5,"ondisk_log_size":5,"up":16,"acting":16,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":74,"ondisk_log_size":74,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359738543,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36528,"kb_used_data":5408,"kb_used_omap":1,"kb_used_meta":31102,"kb_avail":94335312,"statfs":{"total":96636764160,"available":96599359488,"internally_reserved":0,"allocated":5537792,"data_stored":2613966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31848902},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":8,"seq":34359738542,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":69048,"kb_used_data":5992,"kb_used_omap":1,"kb_used_meta":63038,"kb_avail":94302792,"statfs":{"total":96636764160,"available":96566059008,"internally_reserved":0,"allocated":6135808,"data_stored":3204353,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":64551366},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738542,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":63480,"kb_used_data":5992,"kb_used_omap":1,"kb_used_meta":57470,"kb_avail":94308360,"statfs":{"total":96636764160,"available":96571760640,"internally_reserved":0,"allocated":6135808,"data_stored":3204353,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":58849734},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-22T23:17:13.147 DEBUG:teuthology.orchestra.run.trial168:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2026-02-22T23:17:13.312 INFO:teuthology.misc:Shutting down mds daemons... 2026-02-22T23:17:13.312 INFO:teuthology.misc:Shutting down osd daemons... 2026-02-22T23:17:13.312 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2026-02-22T23:17:13.312 INFO:teuthology.orchestra.run:waiting for 300 2026-02-22T23:17:13.359 INFO:tasks.ceph.osd.0:Stopped 2026-02-22T23:17:13.359 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2026-02-22T23:17:13.360 INFO:teuthology.orchestra.run:waiting for 300 2026-02-22T23:17:13.400 INFO:tasks.ceph.osd.1:Stopped 2026-02-22T23:17:13.401 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2026-02-22T23:17:13.401 INFO:teuthology.orchestra.run:waiting for 300 2026-02-22T23:17:13.470 INFO:tasks.ceph.osd.2:Stopped 2026-02-22T23:17:13.470 INFO:teuthology.misc:Shutting down mgr daemons... 2026-02-22T23:17:13.471 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2026-02-22T23:17:13.471 INFO:teuthology.orchestra.run:waiting for 300 2026-02-22T23:17:13.539 INFO:tasks.ceph.mgr.x:Stopped 2026-02-22T23:17:13.539 INFO:teuthology.misc:Shutting down mon daemons... 2026-02-22T23:17:13.539 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2026-02-22T23:17:13.539 INFO:teuthology.orchestra.run:waiting for 300 2026-02-22T23:17:13.589 INFO:tasks.ceph.mon.a:Stopped 2026-02-22T23:17:13.589 INFO:tasks.ceph:Checking cluster log for badness... 2026-02-22T23:17:13.590 DEBUG:teuthology.orchestra.run.trial168:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | head -n 1 2026-02-22T23:17:13.656 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@trial168.front.sepia.ceph.com 2026-02-22T23:17:13.656 DEBUG:teuthology.orchestra.run.trial168:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2026-02-22T23:17:13.726 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@trial168.front.sepia.ceph.com 2026-02-22T23:17:13.727 DEBUG:teuthology.orchestra.run.trial168:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2026-02-22T23:17:13.795 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@trial168.front.sepia.ceph.com 2026-02-22T23:17:13.795 DEBUG:teuthology.orchestra.run.trial168:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2026-02-22T23:17:13.916 INFO:tasks.ceph:Archiving mon data... 2026-02-22T23:17:13.919 DEBUG:teuthology.misc:Transferring archived files from trial168:/var/lib/ceph/mon/ceph-a to /home/teuthworker/mnt/teuthology/yuriw-2026-02-22_19:35:38-rbd-wip-yuri6-testing-2026-02-22-1629-squid-distro-default-trial/64300/data/mon.a.tgz 2026-02-22T23:17:13.920 DEBUG:teuthology.orchestra.run.trial168:> mktemp 2026-02-22T23:17:13.935 INFO:teuthology.orchestra.run.trial168.stdout:/tmp/tmp.jUwPwaFze1 2026-02-22T23:17:13.936 DEBUG:teuthology.orchestra.run.trial168:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.jUwPwaFze1 2026-02-22T23:17:14.045 DEBUG:teuthology.orchestra.run.trial168:> sudo chmod 0666 /tmp/tmp.jUwPwaFze1 2026-02-22T23:17:14.128 DEBUG:teuthology.orchestra.remote:trial168:/tmp/tmp.jUwPwaFze1 is 334KB 2026-02-22T23:17:14.222 DEBUG:teuthology.orchestra.run.trial168:> rm -fr /tmp/tmp.jUwPwaFze1 2026-02-22T23:17:14.241 INFO:tasks.ceph:Cleaning ceph cluster... 2026-02-22T23:17:14.241 DEBUG:teuthology.orchestra.run.trial168:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2026-02-22T23:17:14.360 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2026-02-22T23:17:14.360 INFO:tasks.ceph:Archiving crash dumps... 2026-02-22T23:17:14.363 DEBUG:teuthology.misc:Transferring archived files from trial168:/var/lib/ceph/crash to /home/teuthworker/mnt/teuthology/yuriw-2026-02-22_19:35:38-rbd-wip-yuri6-testing-2026-02-22-1629-squid-distro-default-trial/64300/remote/trial168/crash 2026-02-22T23:17:14.364 DEBUG:teuthology.orchestra.run.trial168:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2026-02-22T23:17:14.424 INFO:tasks.ceph:Compressing logs... 2026-02-22T23:17:14.424 DEBUG:teuthology.orchestra.run.trial168:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2026-02-22T23:17:14.487 INFO:teuthology.orchestra.run.trial168.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2026-02-22T23:17:14.488 INFO:teuthology.orchestra.run.trial168.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36118.log 2026-02-22T23:17:14.488 INFO:teuthology.orchestra.run.trial168.stderr:/var/log/ceph/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39122.log 2026-02-22T23:17:14.489 INFO:teuthology.orchestra.run.trial168.stderr:/var/log/ceph/ceph-client.admin.36118.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2026-02-22T23:17:14.501 INFO:teuthology.orchestra.run.trial168.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36118.log.gz 2026-02-22T23:17:14.501 INFO:teuthology.orchestra.run.trial168.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2026-02-22T23:17:14.502 INFO:teuthology.orchestra.run.trial168.stderr:/var/log/ceph/ceph-client.admin.39122.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39122.log.gz 2026-02-22T23:17:14.502 INFO:teuthology.orchestra.run.trial168.stderr:/var/log/ceph/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36359.log 2026-02-22T23:17:14.502 INFO:teuthology.orchestra.run.trial168.stderr:/var/log/ceph/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39655.log 2026-02-22T23:17:14.502 INFO:teuthology.orchestra.run.trial168.stderr:/var/log/ceph/ceph-client.admin.36359.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36359.log.gz 2026-02-22T23:17:14.502 INFO:teuthology.orchestra.run.trial168.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39097.log 2026-02-22T23:17:14.502 INFO:teuthology.orchestra.run.trial168.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41549.log 2026-02-22T23:17:14.502 INFO:teuthology.orchestra.run.trial168.stderr:/var/log/ceph/ceph-client.admin.39655.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39655.log.gz 2026-02-22T23:17:14.502 INFO:teuthology.orchestra.run.trial168.stderr:/var/log/ceph/ceph-client.admin.39097.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39236.log 2026-02-22T23:17:14.502 INFO:teuthology.orchestra.run.trial168.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39097.log.gz 2026-02-22T23:17:14.502 INFO:teuthology.orchestra.run.trial168.stderr:/var/log/ceph/ceph-client.admin.41549.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41549.log.gz 2026-02-22T23:17:14.502 INFO:teuthology.orchestra.run.trial168.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39285.log 2026-02-22T23:17:14.503 INFO:teuthology.orchestra.run.trial168.stderr:/var/log/ceph/ceph-client.admin.39236.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39187.log 2026-02-22T23:17:14.503 INFO:teuthology.orchestra.run.trial168.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39236.log.gz 2026-02-22T23:17:14.503 INFO:teuthology.orchestra.run.trial168.stderr:/var/log/ceph/ceph-client.admin.39285.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39386.log 2026-02-22T23:17:14.503 INFO:teuthology.orchestra.run.trial168.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39285.log.gz 2026-02-22T23:17:14.503 INFO:teuthology.orchestra.run.trial168.stderr:/var/log/ceph/ceph-client.admin.39187.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39187.log.gz 2026-02-22T23:17:14.503 INFO:teuthology.orchestra.run.trial168.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42041.log 2026-02-22T23:17:14.503 INFO:teuthology.orchestra.run.trial168.stderr:/var/log/ceph/ceph-client.admin.39386.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41990.log 2026-02-22T23:17:14.503 INFO:teuthology.orchestra.run.trial168.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39386.log.gz 2026-02-22T23:17:14.503 INFO:teuthology.orchestra.run.trial168.stderr:/var/log/ceph/ceph-client.admin.42041.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42041.log.gz 2026-02-22T23:17:14.503 INFO:teuthology.orchestra.run.trial168.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41892.log 2026-02-22T23:17:14.503 INFO:teuthology.orchestra.run.trial168.stderr:/var/log/ceph/ceph-client.admin.41990.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41500.log 2026-02-22T23:17:14.503 INFO:teuthology.orchestra.run.trial168.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41990.log.gz 2026-02-22T23:17:14.504 INFO:teuthology.orchestra.run.trial168.stderr:/var/log/ceph/ceph-client.admin.41892.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39387.log 2026-02-22T23:17:14.504 INFO:teuthology.orchestra.run.trial168.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41892.log.gz 2026-02-22T23:17:14.504 INFO:teuthology.orchestra.run.trial168.stderr:/var/log/ceph/ceph-client.admin.41500.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41500.log.gz 2026-02-22T23:17:14.504 INFO:teuthology.orchestra.run.trial168.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38936.log 2026-02-22T23:17:14.504 INFO:teuthology.orchestra.run.trial168.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.39387.log: /var/log/ceph/ceph-client.admin.36292.log 2026-02-22T23:17:14.504 INFO:teuthology.orchestra.run.trial168.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39387.log.gz 2026-02-22T23:17:14.504 INFO:teuthology.orchestra.run.trial168.stderr:/var/log/ceph/ceph-client.admin.38936.log: gzip -5 0.0% --verbose -- replaced with /var/log/ceph/ceph-client.admin.38936.log.gz -- 2026-02-22T23:17:14.504 INFO:teuthology.orchestra.run.trial168.stderr: /var/log/ceph/ceph-client.admin.39388.log 2026-02-22T23:17:14.504 INFO:teuthology.orchestra.run.trial168.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41794.log 2026-02-22T23:17:14.504 INFO:teuthology.orchestra.run.trial168.stderr:/var/log/ceph/ceph-client.admin.36292.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36292.log.gz 2026-02-22T23:17:14.504 INFO:teuthology.orchestra.run.trial168.stderr:/var/log/ceph/ceph-client.admin.39388.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39535.log 2026-02-22T23:17:14.504 INFO:teuthology.orchestra.run.trial168.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39388.log.gz 2026-02-22T23:17:14.504 INFO:teuthology.orchestra.run.trial168.stderr:/var/log/ceph/ceph-client.admin.41794.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39534.log 2026-02-22T23:17:14.505 INFO:teuthology.orchestra.run.trial168.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41794.log.gz 2026-02-22T23:17:14.505 INFO:teuthology.orchestra.run.trial168.stderr:/var/log/ceph/ceph-client.admin.39535.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39535.log.gz 2026-02-22T23:17:14.505 INFO:teuthology.orchestra.run.trial168.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39824.log 2026-02-22T23:17:14.505 INFO:teuthology.orchestra.run.trial168.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39027.log/var/log/ceph/ceph-client.admin.39534.log: 2026-02-22T23:17:14.505 INFO:teuthology.orchestra.run.trial168.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39534.log.gz 2026-02-22T23:17:14.505 INFO:teuthology.orchestra.run.trial168.stderr:/var/log/ceph/ceph-client.admin.39824.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36071.log 2026-02-22T23:17:14.505 INFO:teuthology.orchestra.run.trial168.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39824.log.gz 2026-02-22T23:17:14.505 INFO:teuthology.orchestra.run.trial168.stderr:/var/log/ceph/ceph-client.admin.39027.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2026-02-22T23:17:14.505 INFO:teuthology.orchestra.run.trial168.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39027.log.gz 2026-02-22T23:17:14.505 INFO:teuthology.orchestra.run.trial168.stderr:/var/log/ceph/ceph-client.admin.36071.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36071.log.gz 2026-02-22T23:17:14.505 INFO:teuthology.orchestra.run.trial168.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41647.log 2026-02-22T23:17:14.505 INFO:teuthology.orchestra.run.trial168.stderr:gzip/var/log/ceph/ceph.audit.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.42088.log 2026-02-22T23:17:14.506 INFO:teuthology.orchestra.run.trial168.stderr:/var/log/ceph/ceph-client.admin.41647.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41647.log.gz 2026-02-22T23:17:14.506 INFO:teuthology.orchestra.run.trial168.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.31253.log 2026-02-22T23:17:14.506 INFO:teuthology.orchestra.run.trial168.stderr: 87.6% -- replaced with /var/log/ceph/ceph.audit.log.gz 2026-02-22T23:17:14.506 INFO:teuthology.orchestra.run.trial168.stderr:/var/log/ceph/ceph-client.admin.42088.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41745.log 2026-02-22T23:17:14.506 INFO:teuthology.orchestra.run.trial168.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42088.log.gz 2026-02-22T23:17:14.506 INFO:teuthology.orchestra.run.trial168.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41941.log 2026-02-22T23:17:14.506 INFO:teuthology.orchestra.run.trial168.stderr:/var/log/ceph/ceph.tmp-client.admin.31253.log: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.31253.log.gz 2026-02-22T23:17:14.506 INFO:teuthology.orchestra.run.trial168.stderr:/var/log/ceph/ceph-client.admin.41745.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2026-02-22T23:17:14.506 INFO:teuthology.orchestra.run.trial168.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41745.log.gz 2026-02-22T23:17:14.506 INFO:teuthology.orchestra.run.trial168.stderr:/var/log/ceph/ceph-client.admin.41941.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35886.log 2026-02-22T23:17:14.507 INFO:teuthology.orchestra.run.trial168.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41941.log.gz 2026-02-22T23:17:14.507 INFO:teuthology.orchestra.run.trial168.stderr:gzip/var/log/ceph/ceph-osd.1.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.39654.log 2026-02-22T23:17:14.507 INFO:teuthology.orchestra.run.trial168.stderr:/var/log/ceph/ceph-client.admin.35886.log: gzip 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35886.log.gz -5 2026-02-22T23:17:14.507 INFO:teuthology.orchestra.run.trial168.stderr: --verbose -- /var/log/ceph/ceph-client.admin.39873.log 2026-02-22T23:17:14.507 INFO:teuthology.orchestra.run.trial168.stderr:/var/log/ceph/ceph-client.admin.39654.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41598.log 2026-02-22T23:17:14.507 INFO:teuthology.orchestra.run.trial168.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39654.log.gz 2026-02-22T23:17:14.507 INFO:teuthology.orchestra.run.trial168.stderr:/var/log/ceph/ceph-client.admin.39873.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39686.log 2026-02-22T23:17:14.507 INFO:teuthology.orchestra.run.trial168.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39873.log.gz 2026-02-22T23:17:14.507 INFO:teuthology.orchestra.run.trial168.stderr:/var/log/ceph/ceph-client.admin.41598.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2026-02-22T23:17:14.507 INFO:teuthology.orchestra.run.trial168.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41598.log.gz 2026-02-22T23:17:14.507 INFO:teuthology.orchestra.run.trial168.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36165.log 2026-02-22T23:17:14.507 INFO:teuthology.orchestra.run.trial168.stderr:/var/log/ceph/ceph-client.admin.39686.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39686.log.gz 2026-02-22T23:17:14.507 INFO:teuthology.orchestra.run.trial168.stderr:/var/log/ceph/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39775.log 2026-02-22T23:17:14.508 INFO:teuthology.orchestra.run.trial168.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2026-02-22T23:17:14.508 INFO:teuthology.orchestra.run.trial168.stderr:/var/log/ceph/ceph-client.admin.36165.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36165.log.gz 2026-02-22T23:17:14.508 INFO:teuthology.orchestra.run.trial168.stderr:/var/log/ceph/ceph-client.admin.39775.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40121.log 2026-02-22T23:17:14.508 INFO:teuthology.orchestra.run.trial168.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39775.log.gz 2026-02-22T23:17:14.508 INFO:teuthology.orchestra.run.trial168.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39533.log 2026-02-22T23:17:14.508 INFO:teuthology.orchestra.run.trial168.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41696.log 2026-02-22T23:17:14.508 INFO:teuthology.orchestra.run.trial168.stderr:/var/log/ceph/ceph-client.admin.40121.log: /var/log/ceph/ceph-client.admin.39533.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39533.log.gz 2026-02-22T23:17:14.508 INFO:teuthology.orchestra.run.trial168.stderr: 85.2% -- replaced with /var/log/ceph/ceph-client.admin.40121.log.gz 2026-02-22T23:17:14.508 INFO:teuthology.orchestra.run.trial168.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41843.log 2026-02-22T23:17:14.508 INFO:teuthology.orchestra.run.trial168.stderr:/var/log/ceph/ceph-client.admin.41696.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41696.log.gz 2026-02-22T23:17:14.508 INFO:teuthology.orchestra.run.trial168.stderr:/var/log/ceph/ceph-client.admin.41843.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41843.log.gz 2026-02-22T23:17:14.508 INFO:teuthology.orchestra.run.trial168.stderr: 87.1% -- replaced with /var/log/ceph/ceph.log.gz 2026-02-22T23:17:14.539 INFO:teuthology.orchestra.run.trial168.stderr: 91.6% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2026-02-22T23:17:14.704 INFO:teuthology.orchestra.run.trial168.stderr: 90.1% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2026-02-22T23:17:16.045 INFO:teuthology.orchestra.run.trial168.stderr: 94.8% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2026-02-22T23:17:17.208 INFO:teuthology.orchestra.run.trial168.stderr: 94.7% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2026-02-22T23:17:18.294 INFO:teuthology.orchestra.run.trial168.stderr: 96.2% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2026-02-22T23:17:18.296 INFO:teuthology.orchestra.run.trial168.stderr: 2026-02-22T23:17:18.296 INFO:teuthology.orchestra.run.trial168.stderr:real 0m3.817s 2026-02-22T23:17:18.296 INFO:teuthology.orchestra.run.trial168.stderr:user 0m7.932s 2026-02-22T23:17:18.296 INFO:teuthology.orchestra.run.trial168.stderr:sys 0m0.427s 2026-02-22T23:17:18.296 INFO:tasks.ceph:Archiving logs... 2026-02-22T23:17:18.296 DEBUG:teuthology.misc:Transferring archived files from trial168:/var/log/ceph to /home/teuthworker/mnt/teuthology/yuriw-2026-02-22_19:35:38-rbd-wip-yuri6-testing-2026-02-22-1629-squid-distro-default-trial/64300/remote/trial168/log 2026-02-22T23:17:18.297 DEBUG:teuthology.orchestra.run.trial168:> sudo tar c -f - -C /var/log/ceph -- . 2026-02-22T23:17:18.978 DEBUG:teuthology.run_tasks:Unwinding manager install 2026-02-22T23:17:18.989 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2026-02-22T23:17:18.989 DEBUG:teuthology.orchestra.run.trial168:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2026-02-22T23:17:19.083 DEBUG:teuthology.run_tasks:Unwinding manager clock 2026-02-22T23:17:19.098 INFO:teuthology.task.clock:Checking final clock skew... 2026-02-22T23:17:19.098 DEBUG:teuthology.orchestra.run.trial168:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2026-02-22T23:17:19.113 INFO:teuthology.orchestra.run.trial168.stderr:bash: line 1: ntpq: command not found 2026-02-22T23:17:19.139 INFO:teuthology.orchestra.run.trial168.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2026-02-22T23:17:19.139 INFO:teuthology.orchestra.run.trial168.stdout:=============================================================================== 2026-02-22T23:17:19.139 INFO:teuthology.orchestra.run.trial168.stdout:^* time.cloudflare.com 3 6 377 61 -189us[ -171us] +/- 6482us 2026-02-22T23:17:19.139 INFO:teuthology.orchestra.run.trial168.stdout:^- 23.186.168.130 2 7 367 125 +492us[ +507us] +/- 56ms 2026-02-22T23:17:19.139 INFO:teuthology.orchestra.run.trial168.stdout:^+ 38.45.64.130 2 7 377 126 -851us[ -865us] +/- 18ms 2026-02-22T23:17:19.139 INFO:teuthology.orchestra.run.trial168.stdout:^- ntp.maxhost.io 2 6 277 62 +131us[ +149us] +/- 55ms 2026-02-22T23:17:19.140 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2026-02-22T23:17:19.151 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2026-02-22T23:17:19.151 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2026-02-22T23:17:19.165 DEBUG:teuthology.orchestra.run.trial168:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2026-02-22T23:17:19.232 DEBUG:teuthology.orchestra.run.trial168:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2026-02-22T23:17:19.257 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-22T23:17:19.257 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2026-02-22T23:17:19.274 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2026-02-22T23:17:19.289 INFO:teuthology.task.internal:Duration was 1267.174349 seconds 2026-02-22T23:17:19.289 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2026-02-22T23:17:19.304 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2026-02-22T23:17:19.305 DEBUG:teuthology.orchestra.run.trial168:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2026-02-22T23:17:19.341 INFO:teuthology.orchestra.run.trial168.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2026-02-22T23:17:19.673 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2026-02-22T23:17:19.674 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@trial168.front.sepia.ceph.com 2026-02-22T23:17:19.674 DEBUG:teuthology.orchestra.run.trial168:> grep -E --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | grep -E -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | grep -E -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2026-02-22T23:17:19.732 INFO:teuthology.task.internal.syslog:Gathering journactl... 2026-02-22T23:17:19.732 DEBUG:teuthology.orchestra.run.trial168:> sudo journalctl > /home/ubuntu/cephtest/archive/syslog/journalctl.log 2026-02-22T23:17:19.831 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2026-02-22T23:17:19.831 DEBUG:teuthology.orchestra.run.trial168:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --max-args=1 --max-procs=0 --verbose --no-run-if-empty -- gzip -5 --verbose -- 2026-02-22T23:17:19.854 INFO:teuthology.orchestra.run.trial168.stderr:gzip -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/misc.log 2026-02-22T23:17:19.854 INFO:teuthology.orchestra.run.trial168.stderr:gzip -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/kern.log 2026-02-22T23:17:19.855 INFO:teuthology.orchestra.run.trial168.stderr:/home/ubuntu/cephtest/archive/syslog/misc.log: gzip -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/journalctl.log 2026-02-22T23:17:19.855 INFO:teuthology.orchestra.run.trial168.stderr: 0.0% -- replaced with /home/ubuntu/cephtest/archive/syslog/misc.log.gz 2026-02-22T23:17:19.855 INFO:teuthology.orchestra.run.trial168.stderr:/home/ubuntu/cephtest/archive/syslog/kern.log: 0.0% -- replaced with /home/ubuntu/cephtest/archive/syslog/kern.log.gz 2026-02-22T23:17:19.865 INFO:teuthology.orchestra.run.trial168.stderr:/home/ubuntu/cephtest/archive/syslog/journalctl.log: 81.9% -- replaced with /home/ubuntu/cephtest/archive/syslog/journalctl.log.gz 2026-02-22T23:17:19.867 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2026-02-22T23:17:19.883 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2026-02-22T23:17:19.884 DEBUG:teuthology.orchestra.run.trial168:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2026-02-22T23:17:19.933 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2026-02-22T23:17:19.952 DEBUG:teuthology.orchestra.run.trial168:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2026-02-22T23:17:19.996 INFO:teuthology.orchestra.run.trial168.stdout:kernel.core_pattern = core 2026-02-22T23:17:20.008 DEBUG:teuthology.orchestra.run.trial168:> test -e /home/ubuntu/cephtest/archive/coredump 2026-02-22T23:17:20.063 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-22T23:17:20.064 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2026-02-22T23:17:20.081 INFO:teuthology.task.internal:Transferring archived files... 2026-02-22T23:17:20.082 DEBUG:teuthology.misc:Transferring archived files from trial168:/home/ubuntu/cephtest/archive to /home/teuthworker/mnt/teuthology/yuriw-2026-02-22_19:35:38-rbd-wip-yuri6-testing-2026-02-22-1629-squid-distro-default-trial/64300/remote/trial168 2026-02-22T23:17:20.082 DEBUG:teuthology.orchestra.run.trial168:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2026-02-22T23:17:20.140 INFO:teuthology.task.internal:Removing archive directory... 2026-02-22T23:17:20.140 DEBUG:teuthology.orchestra.run.trial168:> rm -rf -- /home/ubuntu/cephtest/archive 2026-02-22T23:17:20.187 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2026-02-22T23:17:20.205 INFO:teuthology.task.internal:Not uploading archives. 2026-02-22T23:17:20.206 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2026-02-22T23:17:20.222 INFO:teuthology.task.internal:Tidying up after the test... 2026-02-22T23:17:20.222 DEBUG:teuthology.orchestra.run.trial168:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2026-02-22T23:17:20.243 INFO:teuthology.orchestra.run.trial168.stdout: 917572 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Feb 22 23:17 /home/ubuntu/cephtest 2026-02-22T23:17:20.245 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2026-02-22T23:17:20.261 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2026-02-22T23:17:20.303 INFO:teuthology.run:Summary data: description: rbd/valgrind/{base/install centos_latest clusters/{fixed-1 openstack} conf/{disable-pool-app} objectstore/bluestore-hybrid validator/memcheck workloads/python_api_tests} duration: 1267.1743488311768 flavor: default owner: scheduled_yuriw@soko04.front.sepia.ceph.com success: true 2026-02-22T23:17:20.303 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com 2026-02-22T23:17:20.437 INFO:teuthology.run:pass