2026-02-21T23:02:10.471 INFO:root:teuthology version: 1.2.4.dev9+gfa17720d0 2026-02-21T23:02:10.485 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com 2026-02-21T23:02:10.575 INFO:teuthology.run:Config: archive_path: /home/teuthworker/mnt/teuthology/nmordech-2026-02-21_21:55:34-rados-wip-rocky10-branch-of-the-day-2026-02-21-1771692874-tentacle-distro-default-trial/63256 branch: wip-rocky10-branch-of-the-day-2026-02-21-1771692874-tentacle description: rados/singleton-nomsgr/{all/health-warnings mon_election/connectivity rados supported-random-distro$/{centos_latest}} email: null first_in_suite: false flavor: default job_id: '63256' kernel: branch: distro kdb: 1 sha1: distro last_in_suite: false machine_type: trial name: nmordech-2026-02-21_21:55:34-rados-wip-rocky10-branch-of-the-day-2026-02-21-1771692874-tentacle-distro-default-trial no_nested_subset: false os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-rocky10-branch-of-the-day-2026-02-21-1771692874-tentacle ceph: conf: global: mon election default strategy: 3 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 30 debug ms: 1 debug paxos: 20 mon scrub interval: 300 osd: bluestore zero block detection: true debug ms: 1 debug osd: 20 osd debug verify cached snaps: true osd debug verify missing on start: true osd mclock override recovery settings: true osd mclock profile: high_recovery_ops osd mclock skip benchmark: true osd op queue: debug_random osd op queue cut off: debug_random flavor: default log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) sha1: 7c8cf961761b37084e637a305d42f77c52437802 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} cephadm: distro-suffix: rockylinux-10 install: ceph: flavor: default sha1: 7c8cf961761b37084e637a305d42f77c52437802 workunit: branch: main sha1: 53f8441172c46e34523029ab88f9db7a45514e84 owner: scheduled_nmordech@soko04.front.sepia.ceph.com priority: 10 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.x - osd.0 - osd.1 - osd.2 - osd.3 - osd.4 - osd.5 - osd.6 - osd.7 - osd.8 - osd.9 - client.0 seed: 1998 sha1: 7c8cf961761b37084e637a305d42f77c52437802 sleep_before_teardown: 0 subset: 111/120000 suite: rados suite_branch: main suite_path: /home/teuthworker/src/github.com_NitzanMordhai_ceph_53f8441172c46e34523029ab88f9db7a45514e84/qa suite_relpath: qa suite_repo: https://github.com/NitzanMordhai/ceph.git suite_sha1: 53f8441172c46e34523029ab88f9db7a45514e84 targets: trial150.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDfLMdMUlu8wAems/NUkpdaHQ49xQxHGIEb/oAcsKFOv0SadhbslhCgXiykZWo7B1FgOXC2TFblnUWpu2UjoQmI= tasks: - install: null - ceph: conf: osd: osd max object name len: 400 osd max object namespace len: 64 log-ignorelist: - but it is still running - overall HEALTH_ - \(OSDMAP_FLAGS\) - \(OSD_ - \(PG_ - \(POOL_APP_NOT_ENABLED\) pre-mgr-commands: - sudo ceph config set mgr mgr_pool false --force - workunit: clients: all: - rados/test_health_warnings.sh teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: fa17720d0088c3ac28e473468bfc79eeaff5cd38 timestamp: 2026-02-21_21:55:34 tube: trial user: nmordech verbose: false worker_log: /home/teuthworker/mnt/teuthology/worker_logs/dispatcher.trial.1540699 2026-02-21T23:02:10.575 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_NitzanMordhai_ceph_53f8441172c46e34523029ab88f9db7a45514e84/qa; will attempt to use it 2026-02-21T23:02:10.575 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_NitzanMordhai_ceph_53f8441172c46e34523029ab88f9db7a45514e84/qa/tasks 2026-02-21T23:02:10.575 INFO:teuthology.run_tasks:Running task internal.check_packages... 2026-02-21T23:02:10.577 INFO:teuthology.task.internal:Checking packages... 2026-02-21T23:02:10.577 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '7c8cf961761b37084e637a305d42f77c52437802' 2026-02-21T23:02:10.577 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2026-02-21T23:02:10.577 INFO:teuthology.packaging:ref: None 2026-02-21T23:02:10.577 INFO:teuthology.packaging:tag: None 2026-02-21T23:02:10.577 INFO:teuthology.packaging:branch: wip-rocky10-branch-of-the-day-2026-02-21-1771692874-tentacle 2026-02-21T23:02:10.577 INFO:teuthology.packaging:sha1: 7c8cf961761b37084e637a305d42f77c52437802 2026-02-21T23:02:10.577 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-rocky10-branch-of-the-day-2026-02-21-1771692874-tentacle 2026-02-21T23:02:10.706 INFO:teuthology.task.internal:Found packages for ceph version 20.3.0-5425.g7c8cf961 2026-02-21T23:02:10.708 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2026-02-21T23:02:10.720 INFO:teuthology.task.internal:no buildpackages task found 2026-02-21T23:02:10.720 INFO:teuthology.run_tasks:Running task internal.save_config... 2026-02-21T23:02:10.734 INFO:teuthology.task.internal:Saving configuration 2026-02-21T23:02:10.743 INFO:teuthology.run_tasks:Running task internal.check_lock... 2026-02-21T23:02:10.755 INFO:teuthology.task.internal.check_lock:Checking locks... 2026-02-21T23:02:10.832 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'trial150.front.sepia.ceph.com', 'description': '/home/teuthworker/mnt/teuthology/nmordech-2026-02-21_21:55:34-rados-wip-rocky10-branch-of-the-day-2026-02-21-1771692874-tentacle-distro-default-trial/63256', 'up': True, 'machine_type': 'trial', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2026-02-21 22:58:44.377398', 'locked_by': 'scheduled_nmordech@soko04.front.sepia.ceph.com', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDfLMdMUlu8wAems/NUkpdaHQ49xQxHGIEb/oAcsKFOv0SadhbslhCgXiykZWo7B1FgOXC2TFblnUWpu2UjoQmI='} 2026-02-21T23:02:10.832 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2026-02-21T23:02:10.843 INFO:teuthology.task.internal:roles: ubuntu@trial150.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'osd.8', 'osd.9', 'client.0'] 2026-02-21T23:02:10.843 INFO:teuthology.run_tasks:Running task console_log... 2026-02-21T23:02:10.928 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x71f7bcc0db20>, signals=[15]) 2026-02-21T23:02:10.928 INFO:teuthology.run_tasks:Running task internal.connect... 2026-02-21T23:02:10.939 INFO:teuthology.task.internal:Opening connections... 2026-02-21T23:02:10.939 DEBUG:teuthology.task.internal:connecting to ubuntu@trial150.front.sepia.ceph.com 2026-02-21T23:02:10.940 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial150.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2026-02-21T23:02:11.002 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2026-02-21T23:02:11.013 DEBUG:teuthology.orchestra.run.trial150:> uname -m 2026-02-21T23:02:11.074 INFO:teuthology.orchestra.run.trial150.stdout:x86_64 2026-02-21T23:02:11.074 DEBUG:teuthology.orchestra.run.trial150:> cat /etc/os-release 2026-02-21T23:02:11.131 INFO:teuthology.orchestra.run.trial150.stdout:NAME="CentOS Stream" 2026-02-21T23:02:11.131 INFO:teuthology.orchestra.run.trial150.stdout:VERSION="9" 2026-02-21T23:02:11.131 INFO:teuthology.orchestra.run.trial150.stdout:ID="centos" 2026-02-21T23:02:11.131 INFO:teuthology.orchestra.run.trial150.stdout:ID_LIKE="rhel fedora" 2026-02-21T23:02:11.131 INFO:teuthology.orchestra.run.trial150.stdout:VERSION_ID="9" 2026-02-21T23:02:11.131 INFO:teuthology.orchestra.run.trial150.stdout:PLATFORM_ID="platform:el9" 2026-02-21T23:02:11.131 INFO:teuthology.orchestra.run.trial150.stdout:PRETTY_NAME="CentOS Stream 9" 2026-02-21T23:02:11.131 INFO:teuthology.orchestra.run.trial150.stdout:ANSI_COLOR="0;31" 2026-02-21T23:02:11.131 INFO:teuthology.orchestra.run.trial150.stdout:LOGO="fedora-logo-icon" 2026-02-21T23:02:11.131 INFO:teuthology.orchestra.run.trial150.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2026-02-21T23:02:11.131 INFO:teuthology.orchestra.run.trial150.stdout:HOME_URL="https://centos.org/" 2026-02-21T23:02:11.131 INFO:teuthology.orchestra.run.trial150.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2026-02-21T23:02:11.131 INFO:teuthology.orchestra.run.trial150.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2026-02-21T23:02:11.131 INFO:teuthology.orchestra.run.trial150.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2026-02-21T23:02:11.132 INFO:teuthology.lock.ops:Updating trial150.front.sepia.ceph.com on lock server 2026-02-21T23:02:11.222 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2026-02-21T23:02:11.232 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2026-02-21T23:02:11.243 INFO:teuthology.task.internal:Checking for old test directory... 2026-02-21T23:02:11.243 DEBUG:teuthology.orchestra.run.trial150:> test '!' -e /home/ubuntu/cephtest 2026-02-21T23:02:11.258 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2026-02-21T23:02:11.268 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2026-02-21T23:02:11.268 DEBUG:teuthology.orchestra.run.trial150:> test -z $(ls -A /var/lib/ceph) 2026-02-21T23:02:11.315 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2026-02-21T23:02:11.418 INFO:teuthology.run_tasks:Running task kernel... 2026-02-21T23:02:11.441 INFO:teuthology.task.kernel:normalize config orig: {'branch': 'distro', 'kdb': 1, 'sha1': 'distro'} 2026-02-21T23:02:11.441 INFO:teuthology.task.kernel:config {'mon.a': {'branch': 'distro', 'kdb': 1, 'sha1': 'distro'}}, timeout 300 2026-02-21T23:02:11.441 DEBUG:teuthology.orchestra.run.trial150:> test -f /run/.containerenv -o -f /.dockerenv 2026-02-21T23:02:11.456 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-21T23:02:11.456 DEBUG:teuthology.orchestra.run.trial150:> uname -r 2026-02-21T23:02:11.512 INFO:teuthology.orchestra.run.trial150.stdout:5.14.0-681.el9.x86_64 2026-02-21T23:02:11.512 INFO:teuthology.task.kernel:Running kernel on trial150: 5.14.0-681.el9.x86_64 2026-02-21T23:02:11.512 DEBUG:teuthology.orchestra.run.trial150:> sudo yum install -y kernel 2026-02-21T23:02:12.725 INFO:teuthology.orchestra.run.trial150.stdout:CentOS Stream 9 - BaseOS 11 MB/s | 8.9 MB 00:00 2026-02-21T23:02:15.857 INFO:teuthology.orchestra.run.trial150.stdout:CentOS Stream 9 - AppStream 10 MB/s | 27 MB 00:02 2026-02-21T23:02:20.623 INFO:teuthology.orchestra.run.trial150.stdout:CentOS Stream 9 - CRB 3.3 MB/s | 7.8 MB 00:02 2026-02-21T23:02:22.352 INFO:teuthology.orchestra.run.trial150.stdout:CentOS Stream 9 - Extras packages 20 kB/s | 20 kB 00:01 2026-02-21T23:02:22.737 INFO:teuthology.orchestra.run.trial150.stdout:Extra Packages for Enterprise Linux 63 MB/s | 20 MB 00:00 2026-02-21T23:02:26.013 INFO:teuthology.orchestra.run.trial150.stdout:lab-extras 989 kB/s | 50 kB 00:00 2026-02-21T23:02:27.018 INFO:teuthology.orchestra.run.trial150.stdout:Package kernel-5.14.0-645.el9.x86_64 is already installed. 2026-02-21T23:02:27.019 INFO:teuthology.orchestra.run.trial150.stdout:Package kernel-5.14.0-681.el9.x86_64 is already installed. 2026-02-21T23:02:27.037 INFO:teuthology.orchestra.run.trial150.stdout:Dependencies resolved. 2026-02-21T23:02:27.038 INFO:teuthology.orchestra.run.trial150.stdout:Nothing to do. 2026-02-21T23:02:27.038 INFO:teuthology.orchestra.run.trial150.stdout:Complete! 2026-02-21T23:02:27.062 DEBUG:teuthology.orchestra.run.trial150:> echo no | sudo yum reinstall kernel || true 2026-02-21T23:02:27.362 INFO:teuthology.orchestra.run.trial150.stdout:Last metadata expiration check: 0:00:01 ago on Sat 21 Feb 2026 11:02:26 PM UTC. 2026-02-21T23:02:27.442 INFO:teuthology.orchestra.run.trial150.stdout:Dependencies resolved. 2026-02-21T23:02:27.443 INFO:teuthology.orchestra.run.trial150.stdout:================================================================================ 2026-02-21T23:02:27.444 INFO:teuthology.orchestra.run.trial150.stdout: Package Architecture Version Repository Size 2026-02-21T23:02:27.444 INFO:teuthology.orchestra.run.trial150.stdout:================================================================================ 2026-02-21T23:02:27.444 INFO:teuthology.orchestra.run.trial150.stdout:Reinstalling: 2026-02-21T23:02:27.444 INFO:teuthology.orchestra.run.trial150.stdout: kernel x86_64 5.14.0-681.el9 baseos 803 k 2026-02-21T23:02:27.444 INFO:teuthology.orchestra.run.trial150.stdout: 2026-02-21T23:02:27.444 INFO:teuthology.orchestra.run.trial150.stdout:Transaction Summary 2026-02-21T23:02:27.444 INFO:teuthology.orchestra.run.trial150.stdout:================================================================================ 2026-02-21T23:02:27.444 INFO:teuthology.orchestra.run.trial150.stdout: 2026-02-21T23:02:27.444 INFO:teuthology.orchestra.run.trial150.stdout:Total download size: 803 k 2026-02-21T23:02:27.444 INFO:teuthology.orchestra.run.trial150.stdout:Installed size: 0 2026-02-21T23:02:27.445 INFO:teuthology.orchestra.run.trial150.stderr:Operation aborted. 2026-02-21T23:02:27.462 INFO:teuthology.orchestra.run.trial150.stdout:Is this ok [y/N]: 2026-02-21T23:02:27.463 DEBUG:teuthology.orchestra.run.trial150:> sudo yum reinstall -y kernel || true 2026-02-21T23:02:27.762 INFO:teuthology.orchestra.run.trial150.stdout:Last metadata expiration check: 0:00:01 ago on Sat 21 Feb 2026 11:02:26 PM UTC. 2026-02-21T23:02:27.842 INFO:teuthology.orchestra.run.trial150.stdout:Dependencies resolved. 2026-02-21T23:02:27.842 INFO:teuthology.orchestra.run.trial150.stdout:================================================================================ 2026-02-21T23:02:27.842 INFO:teuthology.orchestra.run.trial150.stdout: Package Architecture Version Repository Size 2026-02-21T23:02:27.842 INFO:teuthology.orchestra.run.trial150.stdout:================================================================================ 2026-02-21T23:02:27.843 INFO:teuthology.orchestra.run.trial150.stdout:Reinstalling: 2026-02-21T23:02:27.843 INFO:teuthology.orchestra.run.trial150.stdout: kernel x86_64 5.14.0-681.el9 baseos 803 k 2026-02-21T23:02:27.843 INFO:teuthology.orchestra.run.trial150.stdout: 2026-02-21T23:02:27.843 INFO:teuthology.orchestra.run.trial150.stdout:Transaction Summary 2026-02-21T23:02:27.843 INFO:teuthology.orchestra.run.trial150.stdout:================================================================================ 2026-02-21T23:02:27.843 INFO:teuthology.orchestra.run.trial150.stdout: 2026-02-21T23:02:27.843 INFO:teuthology.orchestra.run.trial150.stdout:Total download size: 803 k 2026-02-21T23:02:27.843 INFO:teuthology.orchestra.run.trial150.stdout:Installed size: 0 2026-02-21T23:02:27.843 INFO:teuthology.orchestra.run.trial150.stdout:Downloading Packages: 2026-02-21T23:02:28.785 INFO:teuthology.orchestra.run.trial150.stdout:kernel-5.14.0-681.el9.x86_64.rpm 910 kB/s | 803 kB 00:00 2026-02-21T23:02:28.785 INFO:teuthology.orchestra.run.trial150.stdout:-------------------------------------------------------------------------------- 2026-02-21T23:02:28.785 INFO:teuthology.orchestra.run.trial150.stdout:Total 852 kB/s | 803 kB 00:00 2026-02-21T23:02:28.795 INFO:teuthology.orchestra.run.trial150.stdout:Running transaction check 2026-02-21T23:02:28.828 INFO:teuthology.orchestra.run.trial150.stdout:Transaction check succeeded. 2026-02-21T23:02:28.828 INFO:teuthology.orchestra.run.trial150.stdout:Running transaction test 2026-02-21T23:02:28.833 INFO:teuthology.orchestra.run.trial150.stdout:Transaction test succeeded. 2026-02-21T23:02:28.833 INFO:teuthology.orchestra.run.trial150.stdout:Running transaction 2026-02-21T23:02:28.857 INFO:teuthology.orchestra.run.trial150.stdout: Preparing : 1/1 2026-02-21T23:02:28.861 INFO:teuthology.orchestra.run.trial150.stdout: Reinstalling : kernel-5.14.0-681.el9.x86_64 1/2 2026-02-21T23:02:28.861 INFO:teuthology.orchestra.run.trial150.stdout: Cleanup : kernel-5.14.0-681.el9.x86_64 2/2 2026-02-21T23:02:28.969 INFO:teuthology.orchestra.run.trial150.stdout: Running scriptlet: kernel-5.14.0-681.el9.x86_64 2/2 2026-02-21T23:02:28.969 INFO:teuthology.orchestra.run.trial150.stdout: Verifying : kernel-5.14.0-681.el9.x86_64 1/2 2026-02-21T23:02:29.024 INFO:teuthology.orchestra.run.trial150.stdout: Verifying : kernel-5.14.0-681.el9.x86_64 2/2 2026-02-21T23:02:29.024 INFO:teuthology.orchestra.run.trial150.stdout: 2026-02-21T23:02:29.024 INFO:teuthology.orchestra.run.trial150.stdout:Reinstalled: 2026-02-21T23:02:29.024 INFO:teuthology.orchestra.run.trial150.stdout: kernel-5.14.0-681.el9.x86_64 2026-02-21T23:02:29.025 INFO:teuthology.orchestra.run.trial150.stdout: 2026-02-21T23:02:29.025 INFO:teuthology.orchestra.run.trial150.stdout:Complete! 2026-02-21T23:02:29.051 DEBUG:teuthology.orchestra.run.trial150:> rpm -q kernel | sort -rV | head -n 1 2026-02-21T23:02:29.076 INFO:teuthology.orchestra.run.trial150.stdout:kernel-5.14.0-681.el9.x86_64 2026-02-21T23:02:29.077 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-681.el9.x86_64 2026-02-21T23:02:29.077 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2026-02-21T23:02:29.077 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2026-02-21T23:02:29.077 DEBUG:teuthology.orchestra.run.trial150:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2026-02-21T23:02:29.141 INFO:teuthology.orchestra.run.trial150.stdout:ttyS1 2026-02-21T23:02:29.177 DEBUG:teuthology.parallel:result is None 2026-02-21T23:02:29.177 INFO:teuthology.run_tasks:Running task internal.base... 2026-02-21T23:02:29.188 INFO:teuthology.task.internal:Creating test directory... 2026-02-21T23:02:29.189 DEBUG:teuthology.orchestra.run.trial150:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2026-02-21T23:02:29.232 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2026-02-21T23:02:29.244 INFO:teuthology.run_tasks:Running task internal.archive... 2026-02-21T23:02:29.256 INFO:teuthology.task.internal:Creating archive directory... 2026-02-21T23:02:29.257 DEBUG:teuthology.orchestra.run.trial150:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2026-02-21T23:02:29.293 INFO:teuthology.run_tasks:Running task internal.coredump... 2026-02-21T23:02:29.309 INFO:teuthology.task.internal:Enabling coredump saving... 2026-02-21T23:02:29.309 DEBUG:teuthology.orchestra.run.trial150:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2026-02-21T23:02:29.352 INFO:teuthology.orchestra.run.trial150.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-21T23:02:29.361 INFO:teuthology.orchestra.run.trial150.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-21T23:02:29.363 INFO:teuthology.run_tasks:Running task internal.sudo... 2026-02-21T23:02:29.374 INFO:teuthology.task.internal:Configuring sudo... 2026-02-21T23:02:29.375 DEBUG:teuthology.orchestra.run.trial150:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2026-02-21T23:02:29.427 INFO:teuthology.run_tasks:Running task internal.syslog... 2026-02-21T23:02:29.443 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2026-02-21T23:02:29.444 DEBUG:teuthology.orchestra.run.trial150:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2026-02-21T23:02:29.484 DEBUG:teuthology.orchestra.run.trial150:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2026-02-21T23:02:29.604 DEBUG:teuthology.orchestra.run.trial150:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2026-02-21T23:02:29.629 DEBUG:teuthology.orchestra.run.trial150:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2026-02-21T23:02:29.763 DEBUG:teuthology.orchestra.run.trial150:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2026-02-21T23:02:29.787 DEBUG:teuthology.orchestra.run.trial150:> set -ex 2026-02-21T23:02:29.787 DEBUG:teuthology.orchestra.run.trial150:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2026-02-21T23:02:29.850 DEBUG:teuthology.orchestra.run.trial150:> sudo service rsyslog restart 2026-02-21T23:02:29.918 INFO:teuthology.orchestra.run.trial150.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2026-02-21T23:02:30.262 INFO:teuthology.run_tasks:Running task internal.timer... 2026-02-21T23:02:30.275 INFO:teuthology.task.internal:Starting timer... 2026-02-21T23:02:30.276 INFO:teuthology.run_tasks:Running task pcp... 2026-02-21T23:02:30.296 INFO:teuthology.run_tasks:Running task selinux... 2026-02-21T23:02:30.314 DEBUG:teuthology.orchestra.run.trial150:> sudo service auditd rotate 2026-02-21T23:02:30.342 INFO:teuthology.orchestra.run.trial150.stdout:Rotating logs: 2026-02-21T23:02:30.344 DEBUG:teuthology.task.selinux:Getting current SELinux state 2026-02-21T23:02:30.344 DEBUG:teuthology.orchestra.run.trial150:> /usr/sbin/getenforce 2026-02-21T23:02:30.401 INFO:teuthology.orchestra.run.trial150.stdout:Permissive 2026-02-21T23:02:30.401 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@trial150.front.sepia.ceph.com': 'permissive'} 2026-02-21T23:02:30.401 DEBUG:teuthology.orchestra.run.trial150:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2026-02-21T23:02:30.465 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-21T23:02:30.465 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2026-02-21T23:02:30.465 DEBUG:teuthology.orchestra.run.trial150:> sudo /usr/sbin/setenforce permissive 2026-02-21T23:02:30.528 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2026-02-21T23:02:30.542 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2026-02-21T23:02:30.545 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2026-02-21T23:02:30.662 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2026-02-21T23:02:30.670 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2026-02-21T23:02:30.671 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit trial150.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2026-02-21T23:05:10.196 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@trial150.front.sepia.ceph.com')] 2026-02-21T23:05:10.198 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@trial150.front.sepia.ceph.com' 2026-02-21T23:05:10.199 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial150.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2026-02-21T23:05:10.311 DEBUG:teuthology.orchestra.run.trial150:> true 2026-02-21T23:05:10.388 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@trial150.front.sepia.ceph.com' 2026-02-21T23:05:10.388 INFO:teuthology.run_tasks:Running task clock... 2026-02-21T23:05:10.404 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2026-02-21T23:05:10.404 INFO:teuthology.orchestra.run:Running command with timeout 360 2026-02-21T23:05:10.404 DEBUG:teuthology.orchestra.run.trial150:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2026-02-21T23:05:10.455 INFO:teuthology.orchestra.run.trial150.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2026-02-21T23:05:10.467 INFO:teuthology.orchestra.run.trial150.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2026-02-21T23:05:10.488 INFO:teuthology.orchestra.run.trial150.stderr:sudo: ntpd: command not found 2026-02-21T23:05:10.497 INFO:teuthology.orchestra.run.trial150.stdout:506 Cannot talk to daemon 2026-02-21T23:05:10.509 INFO:teuthology.orchestra.run.trial150.stderr:Failed to start ntp.service: Unit ntp.service not found. 2026-02-21T23:05:10.520 INFO:teuthology.orchestra.run.trial150.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2026-02-21T23:05:10.572 INFO:teuthology.orchestra.run.trial150.stderr:bash: line 1: ntpq: command not found 2026-02-21T23:05:10.596 INFO:teuthology.orchestra.run.trial150.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2026-02-21T23:05:10.596 INFO:teuthology.orchestra.run.trial150.stdout:=============================================================================== 2026-02-21T23:05:10.596 INFO:teuthology.orchestra.run.trial150.stdout:^? 149.248.12.167.vultruser> 0 6 0 - +0ns[ +0ns] +/- 0ns 2026-02-21T23:05:10.596 INFO:teuthology.orchestra.run.trial150.stdout:^? 139-177-202-26.ip.linode> 0 6 0 - +0ns[ +0ns] +/- 0ns 2026-02-21T23:05:10.596 INFO:teuthology.orchestra.run.trial150.stdout:^? x.ns.gin.ntt.net 0 6 0 - +0ns[ +0ns] +/- 0ns 2026-02-21T23:05:10.596 INFO:teuthology.orchestra.run.trial150.stdout:^? sushi.ruselabs.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2026-02-21T23:05:10.597 INFO:teuthology.run_tasks:Running task install... 2026-02-21T23:05:10.608 DEBUG:teuthology.task.install:project ceph 2026-02-21T23:05:10.608 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '7c8cf961761b37084e637a305d42f77c52437802'}} 2026-02-21T23:05:10.608 DEBUG:teuthology.task.install:config {'flavor': 'default', 'sha1': '7c8cf961761b37084e637a305d42f77c52437802'} 2026-02-21T23:05:10.608 INFO:teuthology.task.install:Using flavor: default 2026-02-21T23:05:10.611 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-osd', 'ceph-osd-classic', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-osd', 'ceph-osd-classic', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2026-02-21T23:05:10.611 INFO:teuthology.task.install:extra packages: [] 2026-02-21T23:05:10.611 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '7c8cf961761b37084e637a305d42f77c52437802', 'tag': None, 'wait_for_package': False} 2026-02-21T23:05:10.612 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=7c8cf961761b37084e637a305d42f77c52437802 2026-02-21T23:05:10.739 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-rocky10-branch-of-the-day-2026-02-21-1771692874-tentacle/7c8cf961761b37084e637a305d42f77c52437802/centos/9/flavors/default/ 2026-02-21T23:05:10.739 INFO:teuthology.task.install.rpm:Package version is 20.3.0-5425.g7c8cf961 2026-02-21T23:05:11.291 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-rocky10-branch-of-the-day-2026-02-21-1771692874-tentacle/7c8cf961761b37084e637a305d42f77c52437802/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-rocky10-branch-of-the-day-2026-02-21-1771692874-tentacle/7c8cf961761b37084e637a305d42f77c52437802/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-rocky10-branch-of-the-day-2026-02-21-1771692874-tentacle/7c8cf961761b37084e637a305d42f77c52437802/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2026-02-21T23:05:11.291 DEBUG:teuthology.orchestra.run.trial150:> set -ex 2026-02-21T23:05:11.291 DEBUG:teuthology.orchestra.run.trial150:> sudo dd of=/etc/yum.repos.d/ceph.repo 2026-02-21T23:05:11.319 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-osd, ceph-osd-classic, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2026-02-21T23:05:11.320 DEBUG:teuthology.orchestra.run.trial150:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/7c8cf961761b37084e637a305d42f77c52437802/;g' /etc/yum.repos.d/ceph.repo ; fi 2026-02-21T23:05:11.385 DEBUG:teuthology.orchestra.run.trial150:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2026-02-21T23:05:11.459 DEBUG:teuthology.orchestra.run.trial150:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2026-02-21T23:05:11.523 INFO:teuthology.orchestra.run.trial150.stdout:check_obsoletes = 1 2026-02-21T23:05:11.525 DEBUG:teuthology.orchestra.run.trial150:> sudo yum clean all 2026-02-21T23:05:11.685 INFO:teuthology.orchestra.run.trial150.stdout:41 files removed 2026-02-21T23:05:11.700 DEBUG:teuthology.orchestra.run.trial150:> sudo yum -y install ceph-radosgw 2026-02-21T23:05:12.512 INFO:teuthology.orchestra.run.trial150.stdout:ceph packages for x86_64 140 kB/s | 93 kB 00:00 2026-02-21T23:05:13.062 INFO:teuthology.orchestra.run.trial150.stdout:ceph noarch packages 47 kB/s | 25 kB 00:00 2026-02-21T23:05:13.321 INFO:teuthology.orchestra.run.trial150.stdout:ceph source packages 8.7 kB/s | 2.1 kB 00:00 2026-02-21T23:05:15.031 INFO:teuthology.orchestra.run.trial150.stdout:CentOS Stream 9 - BaseOS 5.3 MB/s | 8.9 MB 00:01 2026-02-21T23:05:18.282 INFO:teuthology.orchestra.run.trial150.stdout:CentOS Stream 9 - AppStream 9.8 MB/s | 27 MB 00:02 2026-02-21T23:05:21.693 INFO:teuthology.orchestra.run.trial150.stdout:CentOS Stream 9 - CRB 7.5 MB/s | 7.8 MB 00:01 2026-02-21T23:05:22.550 INFO:teuthology.orchestra.run.trial150.stdout:CentOS Stream 9 - Extras packages 152 kB/s | 20 kB 00:00 2026-02-21T23:05:23.388 INFO:teuthology.orchestra.run.trial150.stdout:Extra Packages for Enterprise Linux 26 MB/s | 20 MB 00:00 2026-02-21T23:05:26.597 INFO:teuthology.orchestra.run.trial150.stdout:lab-extras 1.7 MB/s | 50 kB 00:00 2026-02-21T23:05:27.797 INFO:teuthology.orchestra.run.trial150.stdout:Dependencies resolved. 2026-02-21T23:05:27.799 INFO:teuthology.orchestra.run.trial150.stdout:================================================================================ 2026-02-21T23:05:27.799 INFO:teuthology.orchestra.run.trial150.stdout: Package Arch Version Repository Size 2026-02-21T23:05:27.799 INFO:teuthology.orchestra.run.trial150.stdout:================================================================================ 2026-02-21T23:05:27.799 INFO:teuthology.orchestra.run.trial150.stdout:Installing: 2026-02-21T23:05:27.799 INFO:teuthology.orchestra.run.trial150.stdout: ceph-radosgw x86_64 2:20.3.0-5425.g7c8cf961.el9 ceph 13 M 2026-02-21T23:05:27.799 INFO:teuthology.orchestra.run.trial150.stdout:Upgrading: 2026-02-21T23:05:27.799 INFO:teuthology.orchestra.run.trial150.stdout: librados2 x86_64 2:20.3.0-5425.g7c8cf961.el9 ceph 3.5 M 2026-02-21T23:05:27.799 INFO:teuthology.orchestra.run.trial150.stdout: librbd1 x86_64 2:20.3.0-5425.g7c8cf961.el9 ceph 2.8 M 2026-02-21T23:05:27.799 INFO:teuthology.orchestra.run.trial150.stdout:Installing dependencies: 2026-02-21T23:05:27.800 INFO:teuthology.orchestra.run.trial150.stdout: boost-program-options x86_64 1.75.0-13.el9 appstream 104 k 2026-02-21T23:05:27.800 INFO:teuthology.orchestra.run.trial150.stdout: ceph-base x86_64 2:20.3.0-5425.g7c8cf961.el9 ceph 5.2 M 2026-02-21T23:05:27.800 INFO:teuthology.orchestra.run.trial150.stdout: ceph-common x86_64 2:20.3.0-5425.g7c8cf961.el9 ceph 24 M 2026-02-21T23:05:27.800 INFO:teuthology.orchestra.run.trial150.stdout: ceph-selinux x86_64 2:20.3.0-5425.g7c8cf961.el9 ceph 25 k 2026-02-21T23:05:27.800 INFO:teuthology.orchestra.run.trial150.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2026-02-21T23:05:27.800 INFO:teuthology.orchestra.run.trial150.stdout: libarrow x86_64 9.0.0-15.el9 epel 4.4 M 2026-02-21T23:05:27.800 INFO:teuthology.orchestra.run.trial150.stdout: libarrow-doc noarch 9.0.0-15.el9 epel 25 k 2026-02-21T23:05:27.800 INFO:teuthology.orchestra.run.trial150.stdout: libcephfs2 x86_64 2:20.3.0-5425.g7c8cf961.el9 ceph 908 k 2026-02-21T23:05:27.800 INFO:teuthology.orchestra.run.trial150.stdout: libnbd x86_64 1.20.3-4.el9 appstream 164 k 2026-02-21T23:05:27.800 INFO:teuthology.orchestra.run.trial150.stdout: liboath x86_64 2.6.12-1.el9 epel 49 k 2026-02-21T23:05:27.800 INFO:teuthology.orchestra.run.trial150.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2026-02-21T23:05:27.800 INFO:teuthology.orchestra.run.trial150.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2026-02-21T23:05:27.801 INFO:teuthology.orchestra.run.trial150.stdout: libradosstriper1 x86_64 2:20.3.0-5425.g7c8cf961.el9 ceph 248 k 2026-02-21T23:05:27.801 INFO:teuthology.orchestra.run.trial150.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2026-02-21T23:05:27.801 INFO:teuthology.orchestra.run.trial150.stdout: librgw2 x86_64 2:20.3.0-5425.g7c8cf961.el9 ceph 6.5 M 2026-02-21T23:05:27.801 INFO:teuthology.orchestra.run.trial150.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2026-02-21T23:05:27.801 INFO:teuthology.orchestra.run.trial150.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2026-02-21T23:05:27.801 INFO:teuthology.orchestra.run.trial150.stdout: mailcap noarch 2.1.49-5.el9 baseos 33 k 2026-02-21T23:05:27.801 INFO:teuthology.orchestra.run.trial150.stdout: parquet-libs x86_64 9.0.0-15.el9 epel 838 k 2026-02-21T23:05:27.801 INFO:teuthology.orchestra.run.trial150.stdout: pciutils x86_64 3.7.0-7.el9 baseos 93 k 2026-02-21T23:05:27.801 INFO:teuthology.orchestra.run.trial150.stdout: python3-ceph-argparse x86_64 2:20.3.0-5425.g7c8cf961.el9 ceph 45 k 2026-02-21T23:05:27.801 INFO:teuthology.orchestra.run.trial150.stdout: python3-ceph-common x86_64 2:20.3.0-5425.g7c8cf961.el9 ceph 185 k 2026-02-21T23:05:27.801 INFO:teuthology.orchestra.run.trial150.stdout: python3-cephfs x86_64 2:20.3.0-5425.g7c8cf961.el9 ceph 203 k 2026-02-21T23:05:27.801 INFO:teuthology.orchestra.run.trial150.stdout: python3-rados x86_64 2:20.3.0-5425.g7c8cf961.el9 ceph 324 k 2026-02-21T23:05:27.802 INFO:teuthology.orchestra.run.trial150.stdout: python3-rbd x86_64 2:20.3.0-5425.g7c8cf961.el9 ceph 306 k 2026-02-21T23:05:27.802 INFO:teuthology.orchestra.run.trial150.stdout: python3-rgw x86_64 2:20.3.0-5425.g7c8cf961.el9 ceph 100 k 2026-02-21T23:05:27.802 INFO:teuthology.orchestra.run.trial150.stdout: qatlib x86_64 25.08.0-2.el9 appstream 240 k 2026-02-21T23:05:27.802 INFO:teuthology.orchestra.run.trial150.stdout: qatzip-libs x86_64 1.3.1-1.el9 appstream 66 k 2026-02-21T23:05:27.802 INFO:teuthology.orchestra.run.trial150.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2026-02-21T23:05:27.802 INFO:teuthology.orchestra.run.trial150.stdout: thrift x86_64 0.15.0-4.el9 epel 1.6 M 2026-02-21T23:05:27.802 INFO:teuthology.orchestra.run.trial150.stdout:Installing weak dependencies: 2026-02-21T23:05:27.802 INFO:teuthology.orchestra.run.trial150.stdout: libcephfs-daemon x86_64 2:20.3.0-5425.g7c8cf961.el9 ceph 37 k 2026-02-21T23:05:27.802 INFO:teuthology.orchestra.run.trial150.stdout: libcephfs-proxy2 x86_64 2:20.3.0-5425.g7c8cf961.el9 ceph 26 k 2026-02-21T23:05:27.802 INFO:teuthology.orchestra.run.trial150.stdout: nvme-cli x86_64 2.16-1.el9 baseos 1.2 M 2026-02-21T23:05:27.802 INFO:teuthology.orchestra.run.trial150.stdout: qatlib-service x86_64 25.08.0-2.el9 appstream 37 k 2026-02-21T23:05:27.802 INFO:teuthology.orchestra.run.trial150.stdout: 2026-02-21T23:05:27.803 INFO:teuthology.orchestra.run.trial150.stdout:Transaction Summary 2026-02-21T23:05:27.803 INFO:teuthology.orchestra.run.trial150.stdout:================================================================================ 2026-02-21T23:05:27.803 INFO:teuthology.orchestra.run.trial150.stdout:Install 35 Packages 2026-02-21T23:05:27.803 INFO:teuthology.orchestra.run.trial150.stdout:Upgrade 2 Packages 2026-02-21T23:05:27.803 INFO:teuthology.orchestra.run.trial150.stdout: 2026-02-21T23:05:27.803 INFO:teuthology.orchestra.run.trial150.stdout:Total download size: 68 M 2026-02-21T23:05:27.804 INFO:teuthology.orchestra.run.trial150.stdout:Downloading Packages: 2026-02-21T23:05:31.047 INFO:teuthology.orchestra.run.trial150.stdout:(1/37): ceph-base-20.3.0-5425.g7c8cf961.el9.x86 2.0 MB/s | 5.2 MB 00:02 2026-02-21T23:05:31.101 INFO:teuthology.orchestra.run.trial150.stdout:(2/37): ceph-selinux-20.3.0-5425.g7c8cf961.el9. 469 kB/s | 25 kB 00:00 2026-02-21T23:05:31.243 INFO:teuthology.orchestra.run.trial150.stdout:(3/37): libcephfs-daemon-20.3.0-5425.g7c8cf961. 260 kB/s | 37 kB 00:00 2026-02-21T23:05:31.294 INFO:teuthology.orchestra.run.trial150.stdout:(4/37): libcephfs-proxy2-20.3.0-5425.g7c8cf961. 503 kB/s | 26 kB 00:00 2026-02-21T23:05:31.555 INFO:teuthology.orchestra.run.trial150.stdout:(5/37): libcephfs2-20.3.0-5425.g7c8cf961.el9.x8 3.4 MB/s | 908 kB 00:00 2026-02-21T23:05:31.801 INFO:teuthology.orchestra.run.trial150.stdout:(6/37): libradosstriper1-20.3.0-5425.g7c8cf961. 1.0 MB/s | 248 kB 00:00 2026-02-21T23:05:34.650 INFO:teuthology.orchestra.run.trial150.stdout:(7/37): librgw2-20.3.0-5425.g7c8cf961.el9.x86_6 2.3 MB/s | 6.5 MB 00:02 2026-02-21T23:05:34.755 INFO:teuthology.orchestra.run.trial150.stdout:(8/37): python3-ceph-argparse-20.3.0-5425.g7c8c 432 kB/s | 45 kB 00:00 2026-02-21T23:05:34.947 INFO:teuthology.orchestra.run.trial150.stdout:(9/37): python3-ceph-common-20.3.0-5425.g7c8cf9 962 kB/s | 185 kB 00:00 2026-02-21T23:05:35.183 INFO:teuthology.orchestra.run.trial150.stdout:(10/37): python3-cephfs-20.3.0-5425.g7c8cf961.e 861 kB/s | 203 kB 00:00 2026-02-21T23:05:35.391 INFO:teuthology.orchestra.run.trial150.stdout:(11/37): python3-rados-20.3.0-5425.g7c8cf961.el 1.5 MB/s | 324 kB 00:00 2026-02-21T23:05:35.597 INFO:teuthology.orchestra.run.trial150.stdout:(12/37): python3-rbd-20.3.0-5425.g7c8cf961.el9. 1.4 MB/s | 306 kB 00:00 2026-02-21T23:05:35.789 INFO:teuthology.orchestra.run.trial150.stdout:(13/37): python3-rgw-20.3.0-5425.g7c8cf961.el9. 523 kB/s | 100 kB 00:00 2026-02-21T23:05:35.838 INFO:teuthology.orchestra.run.trial150.stdout:(14/37): mailcap-2.1.49-5.el9.noarch.rpm 679 kB/s | 33 kB 00:00 2026-02-21T23:05:35.897 INFO:teuthology.orchestra.run.trial150.stdout:(15/37): nvme-cli-2.16-1.el9.x86_64.rpm 19 MB/s | 1.2 MB 00:00 2026-02-21T23:05:35.908 INFO:teuthology.orchestra.run.trial150.stdout:(16/37): pciutils-3.7.0-7.el9.x86_64.rpm 8.4 MB/s | 93 kB 00:00 2026-02-21T23:05:35.918 INFO:teuthology.orchestra.run.trial150.stdout:(17/37): boost-program-options-1.75.0-13.el9.x8 10 MB/s | 104 kB 00:00 2026-02-21T23:05:35.930 INFO:teuthology.orchestra.run.trial150.stdout:(18/37): libnbd-1.20.3-4.el9.x86_64.rpm 14 MB/s | 164 kB 00:00 2026-02-21T23:05:35.942 INFO:teuthology.orchestra.run.trial150.stdout:(19/37): libpmemobj-1.12.1-1.el9.x86_64.rpm 14 MB/s | 160 kB 00:00 2026-02-21T23:05:35.952 INFO:teuthology.orchestra.run.trial150.stdout:(20/37): librabbitmq-0.11.0-7.el9.x86_64.rpm 4.4 MB/s | 45 kB 00:00 2026-02-21T23:05:35.965 INFO:teuthology.orchestra.run.trial150.stdout:(21/37): librdkafka-1.6.1-102.el9.x86_64.rpm 49 MB/s | 662 kB 00:00 2026-02-21T23:05:35.977 INFO:teuthology.orchestra.run.trial150.stdout:(22/37): lttng-ust-2.12.0-6.el9.x86_64.rpm 24 MB/s | 292 kB 00:00 2026-02-21T23:05:35.989 INFO:teuthology.orchestra.run.trial150.stdout:(23/37): qatlib-25.08.0-2.el9.x86_64.rpm 20 MB/s | 240 kB 00:00 2026-02-21T23:05:35.999 INFO:teuthology.orchestra.run.trial150.stdout:(24/37): qatlib-service-25.08.0-2.el9.x86_64.rp 3.6 MB/s | 37 kB 00:00 2026-02-21T23:05:36.010 INFO:teuthology.orchestra.run.trial150.stdout:(25/37): qatzip-libs-1.3.1-1.el9.x86_64.rpm 6.3 MB/s | 66 kB 00:00 2026-02-21T23:05:36.180 INFO:teuthology.orchestra.run.trial150.stdout:(26/37): gperftools-libs-2.9.1-3.el9.x86_64.rpm 1.8 MB/s | 308 kB 00:00 2026-02-21T23:05:36.296 INFO:teuthology.orchestra.run.trial150.stdout:(27/37): libarrow-9.0.0-15.el9.x86_64.rpm 38 MB/s | 4.4 MB 00:00 2026-02-21T23:05:36.316 INFO:teuthology.orchestra.run.trial150.stdout:(28/37): libarrow-doc-9.0.0-15.el9.noarch.rpm 1.2 MB/s | 25 kB 00:00 2026-02-21T23:05:36.337 INFO:teuthology.orchestra.run.trial150.stdout:(29/37): liboath-2.6.12-1.el9.x86_64.rpm 2.3 MB/s | 49 kB 00:00 2026-02-21T23:05:36.358 INFO:teuthology.orchestra.run.trial150.stdout:(30/37): libunwind-1.6.2-1.el9.x86_64.rpm 3.2 MB/s | 67 kB 00:00 2026-02-21T23:05:36.388 INFO:teuthology.orchestra.run.trial150.stdout:(31/37): parquet-libs-9.0.0-15.el9.x86_64.rpm 27 MB/s | 838 kB 00:00 2026-02-21T23:05:36.410 INFO:teuthology.orchestra.run.trial150.stdout:(32/37): re2-20211101-20.el9.x86_64.rpm 8.5 MB/s | 191 kB 00:00 2026-02-21T23:05:36.449 INFO:teuthology.orchestra.run.trial150.stdout:(33/37): thrift-0.15.0-4.el9.x86_64.rpm 40 MB/s | 1.6 MB 00:00 2026-02-21T23:05:38.052 INFO:teuthology.orchestra.run.trial150.stdout:(34/37): librados2-20.3.0-5425.g7c8cf961.el9.x8 2.2 MB/s | 3.5 MB 00:01 2026-02-21T23:05:38.535 INFO:teuthology.orchestra.run.trial150.stdout:(35/37): ceph-common-20.3.0-5425.g7c8cf961.el9. 2.4 MB/s | 24 MB 00:10 2026-02-21T23:05:38.999 INFO:teuthology.orchestra.run.trial150.stdout:(36/37): librbd1-20.3.0-5425.g7c8cf961.el9.x86_ 3.0 MB/s | 2.8 MB 00:00 2026-02-21T23:05:39.502 INFO:teuthology.orchestra.run.trial150.stdout:(37/37): ceph-radosgw-20.3.0-5425.g7c8cf961.el9 1.2 MB/s | 13 MB 00:11 2026-02-21T23:05:39.505 INFO:teuthology.orchestra.run.trial150.stdout:-------------------------------------------------------------------------------- 2026-02-21T23:05:39.505 INFO:teuthology.orchestra.run.trial150.stdout:Total 5.8 MB/s | 68 MB 00:11 2026-02-21T23:05:39.606 INFO:teuthology.orchestra.run.trial150.stdout:Running transaction check 2026-02-21T23:05:39.625 INFO:teuthology.orchestra.run.trial150.stdout:Transaction check succeeded. 2026-02-21T23:05:39.626 INFO:teuthology.orchestra.run.trial150.stdout:Running transaction test 2026-02-21T23:05:39.896 INFO:teuthology.orchestra.run.trial150.stdout:Transaction test succeeded. 2026-02-21T23:05:39.897 INFO:teuthology.orchestra.run.trial150.stdout:Running transaction 2026-02-21T23:05:40.206 INFO:teuthology.orchestra.run.trial150.stdout: Preparing : 1/1 2026-02-21T23:05:40.358 INFO:teuthology.orchestra.run.trial150.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/39 2026-02-21T23:05:40.362 INFO:teuthology.orchestra.run.trial150.stdout: Upgrading : librados2-2:20.3.0-5425.g7c8cf961.el9.x86_64 2/39 2026-02-21T23:05:40.414 INFO:teuthology.orchestra.run.trial150.stdout: Running scriptlet: librados2-2:20.3.0-5425.g7c8cf961.el9.x86_64 2/39 2026-02-21T23:05:40.415 INFO:teuthology.orchestra.run.trial150.stdout: Installing : libcephfs2-2:20.3.0-5425.g7c8cf961.el9.x86_64 3/39 2026-02-21T23:05:40.428 INFO:teuthology.orchestra.run.trial150.stdout: Running scriptlet: libcephfs2-2:20.3.0-5425.g7c8cf961.el9.x86_64 3/39 2026-02-21T23:05:40.430 INFO:teuthology.orchestra.run.trial150.stdout: Installing : libcephfs-daemon-2:20.3.0-5425.g7c8cf961.el9.x86_6 4/39 2026-02-21T23:05:40.431 INFO:teuthology.orchestra.run.trial150.stdout: Installing : libcephfs-proxy2-2:20.3.0-5425.g7c8cf961.el9.x86_6 5/39 2026-02-21T23:05:40.458 INFO:teuthology.orchestra.run.trial150.stdout: Running scriptlet: libcephfs-proxy2-2:20.3.0-5425.g7c8cf961.el9.x86_6 5/39 2026-02-21T23:05:40.460 INFO:teuthology.orchestra.run.trial150.stdout: Installing : python3-rados-2:20.3.0-5425.g7c8cf961.el9.x86_64 6/39 2026-02-21T23:05:40.468 INFO:teuthology.orchestra.run.trial150.stdout: Installing : liboath-2.6.12-1.el9.x86_64 7/39 2026-02-21T23:05:40.471 INFO:teuthology.orchestra.run.trial150.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 8/39 2026-02-21T23:05:40.474 INFO:teuthology.orchestra.run.trial150.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 9/39 2026-02-21T23:05:40.478 INFO:teuthology.orchestra.run.trial150.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 10/39 2026-02-21T23:05:40.489 INFO:teuthology.orchestra.run.trial150.stdout: Installing : python3-ceph-argparse-2:20.3.0-5425.g7c8cf961.el9. 11/39 2026-02-21T23:05:40.500 INFO:teuthology.orchestra.run.trial150.stdout: Installing : python3-cephfs-2:20.3.0-5425.g7c8cf961.el9.x86_64 12/39 2026-02-21T23:05:40.501 INFO:teuthology.orchestra.run.trial150.stdout: Installing : libradosstriper1-2:20.3.0-5425.g7c8cf961.el9.x86_6 13/39 2026-02-21T23:05:40.526 INFO:teuthology.orchestra.run.trial150.stdout: Running scriptlet: libradosstriper1-2:20.3.0-5425.g7c8cf961.el9.x86_6 13/39 2026-02-21T23:05:40.530 INFO:teuthology.orchestra.run.trial150.stdout: Installing : thrift-0.15.0-4.el9.x86_64 14/39 2026-02-21T23:05:40.567 INFO:teuthology.orchestra.run.trial150.stdout: Installing : re2-1:20211101-20.el9.x86_64 15/39 2026-02-21T23:05:40.571 INFO:teuthology.orchestra.run.trial150.stdout: Installing : libarrow-9.0.0-15.el9.x86_64 16/39 2026-02-21T23:05:40.577 INFO:teuthology.orchestra.run.trial150.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 17/39 2026-02-21T23:05:40.580 INFO:teuthology.orchestra.run.trial150.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 18/39 2026-02-21T23:05:40.585 INFO:teuthology.orchestra.run.trial150.stdout: Installing : libarrow-doc-9.0.0-15.el9.noarch 19/39 2026-02-21T23:05:40.707 INFO:teuthology.orchestra.run.trial150.stdout: Installing : libnbd-1.20.3-4.el9.x86_64 20/39 2026-02-21T23:05:40.708 INFO:teuthology.orchestra.run.trial150.stdout: Upgrading : librbd1-2:20.3.0-5425.g7c8cf961.el9.x86_64 21/39 2026-02-21T23:05:40.734 INFO:teuthology.orchestra.run.trial150.stdout: Running scriptlet: librbd1-2:20.3.0-5425.g7c8cf961.el9.x86_64 21/39 2026-02-21T23:05:40.736 INFO:teuthology.orchestra.run.trial150.stdout: Installing : python3-rbd-2:20.3.0-5425.g7c8cf961.el9.x86_64 22/39 2026-02-21T23:05:40.742 INFO:teuthology.orchestra.run.trial150.stdout: Installing : boost-program-options-1.75.0-13.el9.x86_64 23/39 2026-02-21T23:05:41.024 INFO:teuthology.orchestra.run.trial150.stdout: Installing : parquet-libs-9.0.0-15.el9.x86_64 24/39 2026-02-21T23:05:41.025 INFO:teuthology.orchestra.run.trial150.stdout: Installing : librgw2-2:20.3.0-5425.g7c8cf961.el9.x86_64 25/39 2026-02-21T23:05:41.042 INFO:teuthology.orchestra.run.trial150.stdout: Running scriptlet: librgw2-2:20.3.0-5425.g7c8cf961.el9.x86_64 25/39 2026-02-21T23:05:41.046 INFO:teuthology.orchestra.run.trial150.stdout: Installing : python3-rgw-2:20.3.0-5425.g7c8cf961.el9.x86_64 26/39 2026-02-21T23:05:41.052 INFO:teuthology.orchestra.run.trial150.stdout: Installing : pciutils-3.7.0-7.el9.x86_64 27/39 2026-02-21T23:05:41.055 INFO:teuthology.orchestra.run.trial150.stdout: Installing : qatlib-25.08.0-2.el9.x86_64 28/39 2026-02-21T23:05:41.055 INFO:teuthology.orchestra.run.trial150.stdout: Installing : qatlib-service-25.08.0-2.el9.x86_64 29/39 2026-02-21T23:05:41.074 INFO:teuthology.orchestra.run.trial150.stdout: Running scriptlet: qatlib-service-25.08.0-2.el9.x86_64 29/39 2026-02-21T23:05:41.191 INFO:teuthology.orchestra.run.trial150.stdout: Installing : qatzip-libs-1.3.1-1.el9.x86_64 30/39 2026-02-21T23:05:41.193 INFO:teuthology.orchestra.run.trial150.stdout: Installing : nvme-cli-2.16-1.el9.x86_64 31/39 2026-02-21T23:05:41.482 INFO:teuthology.orchestra.run.trial150.stdout: Running scriptlet: nvme-cli-2.16-1.el9.x86_64 31/39 2026-02-21T23:05:41.483 INFO:teuthology.orchestra.run.trial150.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service → /usr/lib/systemd/system/nvmefc-boot-connections.service. 2026-02-21T23:05:41.483 INFO:teuthology.orchestra.run.trial150.stdout: 2026-02-21T23:05:41.526 INFO:teuthology.orchestra.run.trial150.stdout: Installing : mailcap-2.1.49-5.el9.noarch 32/39 2026-02-21T23:05:41.528 INFO:teuthology.orchestra.run.trial150.stdout: Installing : python3-ceph-common-2:20.3.0-5425.g7c8cf961.el9.x8 33/39 2026-02-21T23:05:42.617 INFO:teuthology.orchestra.run.trial150.stdout: Running scriptlet: ceph-common-2:20.3.0-5425.g7c8cf961.el9.x86_64 34/39 2026-02-21T23:05:42.619 INFO:teuthology.orchestra.run.trial150.stdout: Installing : ceph-common-2:20.3.0-5425.g7c8cf961.el9.x86_64 34/39 2026-02-21T23:05:42.885 INFO:teuthology.orchestra.run.trial150.stdout: Running scriptlet: ceph-common-2:20.3.0-5425.g7c8cf961.el9.x86_64 34/39 2026-02-21T23:05:42.888 INFO:teuthology.orchestra.run.trial150.stdout: Installing : ceph-base-2:20.3.0-5425.g7c8cf961.el9.x86_64 35/39 2026-02-21T23:05:42.929 INFO:teuthology.orchestra.run.trial150.stdout: Running scriptlet: ceph-base-2:20.3.0-5425.g7c8cf961.el9.x86_64 35/39 2026-02-21T23:05:42.930 INFO:teuthology.orchestra.run.trial150.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2026-02-21T23:05:42.930 INFO:teuthology.orchestra.run.trial150.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2026-02-21T23:05:42.930 INFO:teuthology.orchestra.run.trial150.stdout: 2026-02-21T23:05:42.932 INFO:teuthology.orchestra.run.trial150.stdout: Installing : ceph-selinux-2:20.3.0-5425.g7c8cf961.el9.x86_64 36/39 2026-02-21T23:05:47.525 INFO:teuthology.orchestra.run.trial150.stdout: Running scriptlet: ceph-selinux-2:20.3.0-5425.g7c8cf961.el9.x86_64 36/39 2026-02-21T23:05:47.525 INFO:teuthology.orchestra.run.trial150.stdout:skipping the directory /sys 2026-02-21T23:05:47.525 INFO:teuthology.orchestra.run.trial150.stdout:skipping the directory /proc 2026-02-21T23:05:47.526 INFO:teuthology.orchestra.run.trial150.stdout:skipping the directory /mnt 2026-02-21T23:05:47.526 INFO:teuthology.orchestra.run.trial150.stdout:skipping the directory /var/tmp 2026-02-21T23:05:47.526 INFO:teuthology.orchestra.run.trial150.stdout:skipping the directory /home 2026-02-21T23:05:47.526 INFO:teuthology.orchestra.run.trial150.stdout:skipping the directory /root 2026-02-21T23:05:47.526 INFO:teuthology.orchestra.run.trial150.stdout:skipping the directory /tmp 2026-02-21T23:05:47.526 INFO:teuthology.orchestra.run.trial150.stdout: 2026-02-21T23:05:48.107 INFO:teuthology.orchestra.run.trial150.stdout: Installing : ceph-radosgw-2:20.3.0-5425.g7c8cf961.el9.x86_64 37/39 2026-02-21T23:05:48.130 INFO:teuthology.orchestra.run.trial150.stdout: Running scriptlet: ceph-radosgw-2:20.3.0-5425.g7c8cf961.el9.x86_64 37/39 2026-02-21T23:05:48.130 INFO:teuthology.orchestra.run.trial150.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-21T23:05:48.130 INFO:teuthology.orchestra.run.trial150.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2026-02-21T23:05:48.130 INFO:teuthology.orchestra.run.trial150.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2026-02-21T23:05:48.130 INFO:teuthology.orchestra.run.trial150.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2026-02-21T23:05:48.130 INFO:teuthology.orchestra.run.trial150.stdout: 2026-02-21T23:05:48.132 INFO:teuthology.orchestra.run.trial150.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 38/39 2026-02-21T23:05:48.145 INFO:teuthology.orchestra.run.trial150.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 38/39 2026-02-21T23:05:48.145 INFO:teuthology.orchestra.run.trial150.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 39/39 2026-02-21T23:05:48.201 INFO:teuthology.orchestra.run.trial150.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 39/39 2026-02-21T23:05:48.201 INFO:teuthology.orchestra.run.trial150.stdout:Creating group 'qat' with GID 992. 2026-02-21T23:05:48.201 INFO:teuthology.orchestra.run.trial150.stdout: 2026-02-21T23:05:48.507 INFO:teuthology.orchestra.run.trial150.stdout: Verifying : ceph-base-2:20.3.0-5425.g7c8cf961.el9.x86_64 1/39 2026-02-21T23:05:48.507 INFO:teuthology.orchestra.run.trial150.stdout: Verifying : ceph-common-2:20.3.0-5425.g7c8cf961.el9.x86_64 2/39 2026-02-21T23:05:48.507 INFO:teuthology.orchestra.run.trial150.stdout: Verifying : ceph-radosgw-2:20.3.0-5425.g7c8cf961.el9.x86_64 3/39 2026-02-21T23:05:48.507 INFO:teuthology.orchestra.run.trial150.stdout: Verifying : ceph-selinux-2:20.3.0-5425.g7c8cf961.el9.x86_64 4/39 2026-02-21T23:05:48.507 INFO:teuthology.orchestra.run.trial150.stdout: Verifying : libcephfs-daemon-2:20.3.0-5425.g7c8cf961.el9.x86_6 5/39 2026-02-21T23:05:48.507 INFO:teuthology.orchestra.run.trial150.stdout: Verifying : libcephfs-proxy2-2:20.3.0-5425.g7c8cf961.el9.x86_6 6/39 2026-02-21T23:05:48.507 INFO:teuthology.orchestra.run.trial150.stdout: Verifying : libcephfs2-2:20.3.0-5425.g7c8cf961.el9.x86_64 7/39 2026-02-21T23:05:48.507 INFO:teuthology.orchestra.run.trial150.stdout: Verifying : libradosstriper1-2:20.3.0-5425.g7c8cf961.el9.x86_6 8/39 2026-02-21T23:05:48.507 INFO:teuthology.orchestra.run.trial150.stdout: Verifying : librgw2-2:20.3.0-5425.g7c8cf961.el9.x86_64 9/39 2026-02-21T23:05:48.507 INFO:teuthology.orchestra.run.trial150.stdout: Verifying : python3-ceph-argparse-2:20.3.0-5425.g7c8cf961.el9. 10/39 2026-02-21T23:05:48.508 INFO:teuthology.orchestra.run.trial150.stdout: Verifying : python3-ceph-common-2:20.3.0-5425.g7c8cf961.el9.x8 11/39 2026-02-21T23:05:48.508 INFO:teuthology.orchestra.run.trial150.stdout: Verifying : python3-cephfs-2:20.3.0-5425.g7c8cf961.el9.x86_64 12/39 2026-02-21T23:05:48.508 INFO:teuthology.orchestra.run.trial150.stdout: Verifying : python3-rados-2:20.3.0-5425.g7c8cf961.el9.x86_64 13/39 2026-02-21T23:05:48.508 INFO:teuthology.orchestra.run.trial150.stdout: Verifying : python3-rbd-2:20.3.0-5425.g7c8cf961.el9.x86_64 14/39 2026-02-21T23:05:48.508 INFO:teuthology.orchestra.run.trial150.stdout: Verifying : python3-rgw-2:20.3.0-5425.g7c8cf961.el9.x86_64 15/39 2026-02-21T23:05:48.508 INFO:teuthology.orchestra.run.trial150.stdout: Verifying : mailcap-2.1.49-5.el9.noarch 16/39 2026-02-21T23:05:48.508 INFO:teuthology.orchestra.run.trial150.stdout: Verifying : nvme-cli-2.16-1.el9.x86_64 17/39 2026-02-21T23:05:48.508 INFO:teuthology.orchestra.run.trial150.stdout: Verifying : pciutils-3.7.0-7.el9.x86_64 18/39 2026-02-21T23:05:48.508 INFO:teuthology.orchestra.run.trial150.stdout: Verifying : boost-program-options-1.75.0-13.el9.x86_64 19/39 2026-02-21T23:05:48.508 INFO:teuthology.orchestra.run.trial150.stdout: Verifying : libnbd-1.20.3-4.el9.x86_64 20/39 2026-02-21T23:05:48.508 INFO:teuthology.orchestra.run.trial150.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 21/39 2026-02-21T23:05:48.508 INFO:teuthology.orchestra.run.trial150.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 22/39 2026-02-21T23:05:48.508 INFO:teuthology.orchestra.run.trial150.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 23/39 2026-02-21T23:05:48.508 INFO:teuthology.orchestra.run.trial150.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 24/39 2026-02-21T23:05:48.508 INFO:teuthology.orchestra.run.trial150.stdout: Verifying : qatlib-25.08.0-2.el9.x86_64 25/39 2026-02-21T23:05:48.508 INFO:teuthology.orchestra.run.trial150.stdout: Verifying : qatlib-service-25.08.0-2.el9.x86_64 26/39 2026-02-21T23:05:48.508 INFO:teuthology.orchestra.run.trial150.stdout: Verifying : qatzip-libs-1.3.1-1.el9.x86_64 27/39 2026-02-21T23:05:48.508 INFO:teuthology.orchestra.run.trial150.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 28/39 2026-02-21T23:05:48.508 INFO:teuthology.orchestra.run.trial150.stdout: Verifying : libarrow-9.0.0-15.el9.x86_64 29/39 2026-02-21T23:05:48.508 INFO:teuthology.orchestra.run.trial150.stdout: Verifying : libarrow-doc-9.0.0-15.el9.noarch 30/39 2026-02-21T23:05:48.508 INFO:teuthology.orchestra.run.trial150.stdout: Verifying : liboath-2.6.12-1.el9.x86_64 31/39 2026-02-21T23:05:48.508 INFO:teuthology.orchestra.run.trial150.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 32/39 2026-02-21T23:05:48.508 INFO:teuthology.orchestra.run.trial150.stdout: Verifying : parquet-libs-9.0.0-15.el9.x86_64 33/39 2026-02-21T23:05:48.508 INFO:teuthology.orchestra.run.trial150.stdout: Verifying : re2-1:20211101-20.el9.x86_64 34/39 2026-02-21T23:05:48.508 INFO:teuthology.orchestra.run.trial150.stdout: Verifying : thrift-0.15.0-4.el9.x86_64 35/39 2026-02-21T23:05:48.508 INFO:teuthology.orchestra.run.trial150.stdout: Verifying : librados2-2:20.3.0-5425.g7c8cf961.el9.x86_64 36/39 2026-02-21T23:05:48.508 INFO:teuthology.orchestra.run.trial150.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 37/39 2026-02-21T23:05:48.508 INFO:teuthology.orchestra.run.trial150.stdout: Verifying : librbd1-2:20.3.0-5425.g7c8cf961.el9.x86_64 38/39 2026-02-21T23:05:48.571 INFO:teuthology.orchestra.run.trial150.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 39/39 2026-02-21T23:05:48.571 INFO:teuthology.orchestra.run.trial150.stdout: 2026-02-21T23:05:48.571 INFO:teuthology.orchestra.run.trial150.stdout:Upgraded: 2026-02-21T23:05:48.571 INFO:teuthology.orchestra.run.trial150.stdout: librados2-2:20.3.0-5425.g7c8cf961.el9.x86_64 2026-02-21T23:05:48.572 INFO:teuthology.orchestra.run.trial150.stdout: librbd1-2:20.3.0-5425.g7c8cf961.el9.x86_64 2026-02-21T23:05:48.572 INFO:teuthology.orchestra.run.trial150.stdout:Installed: 2026-02-21T23:05:48.572 INFO:teuthology.orchestra.run.trial150.stdout: boost-program-options-1.75.0-13.el9.x86_64 2026-02-21T23:05:48.572 INFO:teuthology.orchestra.run.trial150.stdout: ceph-base-2:20.3.0-5425.g7c8cf961.el9.x86_64 2026-02-21T23:05:48.572 INFO:teuthology.orchestra.run.trial150.stdout: ceph-common-2:20.3.0-5425.g7c8cf961.el9.x86_64 2026-02-21T23:05:48.572 INFO:teuthology.orchestra.run.trial150.stdout: ceph-radosgw-2:20.3.0-5425.g7c8cf961.el9.x86_64 2026-02-21T23:05:48.572 INFO:teuthology.orchestra.run.trial150.stdout: ceph-selinux-2:20.3.0-5425.g7c8cf961.el9.x86_64 2026-02-21T23:05:48.572 INFO:teuthology.orchestra.run.trial150.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2026-02-21T23:05:48.572 INFO:teuthology.orchestra.run.trial150.stdout: libarrow-9.0.0-15.el9.x86_64 2026-02-21T23:05:48.572 INFO:teuthology.orchestra.run.trial150.stdout: libarrow-doc-9.0.0-15.el9.noarch 2026-02-21T23:05:48.572 INFO:teuthology.orchestra.run.trial150.stdout: libcephfs-daemon-2:20.3.0-5425.g7c8cf961.el9.x86_64 2026-02-21T23:05:48.572 INFO:teuthology.orchestra.run.trial150.stdout: libcephfs-proxy2-2:20.3.0-5425.g7c8cf961.el9.x86_64 2026-02-21T23:05:48.572 INFO:teuthology.orchestra.run.trial150.stdout: libcephfs2-2:20.3.0-5425.g7c8cf961.el9.x86_64 2026-02-21T23:05:48.572 INFO:teuthology.orchestra.run.trial150.stdout: libnbd-1.20.3-4.el9.x86_64 2026-02-21T23:05:48.572 INFO:teuthology.orchestra.run.trial150.stdout: liboath-2.6.12-1.el9.x86_64 2026-02-21T23:05:48.572 INFO:teuthology.orchestra.run.trial150.stdout: libpmemobj-1.12.1-1.el9.x86_64 2026-02-21T23:05:48.572 INFO:teuthology.orchestra.run.trial150.stdout: librabbitmq-0.11.0-7.el9.x86_64 2026-02-21T23:05:48.572 INFO:teuthology.orchestra.run.trial150.stdout: libradosstriper1-2:20.3.0-5425.g7c8cf961.el9.x86_64 2026-02-21T23:05:48.572 INFO:teuthology.orchestra.run.trial150.stdout: librdkafka-1.6.1-102.el9.x86_64 2026-02-21T23:05:48.572 INFO:teuthology.orchestra.run.trial150.stdout: librgw2-2:20.3.0-5425.g7c8cf961.el9.x86_64 2026-02-21T23:05:48.572 INFO:teuthology.orchestra.run.trial150.stdout: libunwind-1.6.2-1.el9.x86_64 2026-02-21T23:05:48.572 INFO:teuthology.orchestra.run.trial150.stdout: lttng-ust-2.12.0-6.el9.x86_64 2026-02-21T23:05:48.573 INFO:teuthology.orchestra.run.trial150.stdout: mailcap-2.1.49-5.el9.noarch 2026-02-21T23:05:48.573 INFO:teuthology.orchestra.run.trial150.stdout: nvme-cli-2.16-1.el9.x86_64 2026-02-21T23:05:48.573 INFO:teuthology.orchestra.run.trial150.stdout: parquet-libs-9.0.0-15.el9.x86_64 2026-02-21T23:05:48.573 INFO:teuthology.orchestra.run.trial150.stdout: pciutils-3.7.0-7.el9.x86_64 2026-02-21T23:05:48.573 INFO:teuthology.orchestra.run.trial150.stdout: python3-ceph-argparse-2:20.3.0-5425.g7c8cf961.el9.x86_64 2026-02-21T23:05:48.573 INFO:teuthology.orchestra.run.trial150.stdout: python3-ceph-common-2:20.3.0-5425.g7c8cf961.el9.x86_64 2026-02-21T23:05:48.573 INFO:teuthology.orchestra.run.trial150.stdout: python3-cephfs-2:20.3.0-5425.g7c8cf961.el9.x86_64 2026-02-21T23:05:48.573 INFO:teuthology.orchestra.run.trial150.stdout: python3-rados-2:20.3.0-5425.g7c8cf961.el9.x86_64 2026-02-21T23:05:48.573 INFO:teuthology.orchestra.run.trial150.stdout: python3-rbd-2:20.3.0-5425.g7c8cf961.el9.x86_64 2026-02-21T23:05:48.573 INFO:teuthology.orchestra.run.trial150.stdout: python3-rgw-2:20.3.0-5425.g7c8cf961.el9.x86_64 2026-02-21T23:05:48.573 INFO:teuthology.orchestra.run.trial150.stdout: qatlib-25.08.0-2.el9.x86_64 2026-02-21T23:05:48.573 INFO:teuthology.orchestra.run.trial150.stdout: qatlib-service-25.08.0-2.el9.x86_64 2026-02-21T23:05:48.573 INFO:teuthology.orchestra.run.trial150.stdout: qatzip-libs-1.3.1-1.el9.x86_64 2026-02-21T23:05:48.573 INFO:teuthology.orchestra.run.trial150.stdout: re2-1:20211101-20.el9.x86_64 2026-02-21T23:05:48.573 INFO:teuthology.orchestra.run.trial150.stdout: thrift-0.15.0-4.el9.x86_64 2026-02-21T23:05:48.573 INFO:teuthology.orchestra.run.trial150.stdout: 2026-02-21T23:05:48.573 INFO:teuthology.orchestra.run.trial150.stdout:Complete! 2026-02-21T23:05:48.614 DEBUG:teuthology.orchestra.run.trial150:> sudo yum -y install ceph-test 2026-02-21T23:05:48.927 INFO:teuthology.orchestra.run.trial150.stdout:Last metadata expiration check: 0:00:22 ago on Sat 21 Feb 2026 11:05:26 PM UTC. 2026-02-21T23:05:49.010 INFO:teuthology.orchestra.run.trial150.stdout:Dependencies resolved. 2026-02-21T23:05:49.010 INFO:teuthology.orchestra.run.trial150.stdout:================================================================================ 2026-02-21T23:05:49.010 INFO:teuthology.orchestra.run.trial150.stdout: Package Arch Version Repository Size 2026-02-21T23:05:49.010 INFO:teuthology.orchestra.run.trial150.stdout:================================================================================ 2026-02-21T23:05:49.010 INFO:teuthology.orchestra.run.trial150.stdout:Installing: 2026-02-21T23:05:49.010 INFO:teuthology.orchestra.run.trial150.stdout: ceph-test x86_64 2:20.3.0-5425.g7c8cf961.el9 ceph 78 M 2026-02-21T23:05:49.010 INFO:teuthology.orchestra.run.trial150.stdout:Installing dependencies: 2026-02-21T23:05:49.010 INFO:teuthology.orchestra.run.trial150.stdout: hwloc-libs x86_64 2.4.1-6.el9 baseos 2.1 M 2026-02-21T23:05:49.011 INFO:teuthology.orchestra.run.trial150.stdout: libatomic x86_64 11.5.0-14.el9 baseos 23 k 2026-02-21T23:05:49.011 INFO:teuthology.orchestra.run.trial150.stdout: libcephsqlite x86_64 2:20.3.0-5425.g7c8cf961.el9 ceph 159 k 2026-02-21T23:05:49.011 INFO:teuthology.orchestra.run.trial150.stdout: libxslt x86_64 1.1.34-12.el9 appstream 233 k 2026-02-21T23:05:49.011 INFO:teuthology.orchestra.run.trial150.stdout: lksctp-tools x86_64 1.0.19-2.el9 baseos 94 k 2026-02-21T23:05:49.011 INFO:teuthology.orchestra.run.trial150.stdout: protobuf x86_64 3.14.0-17.el9 appstream 1.0 M 2026-02-21T23:05:49.011 INFO:teuthology.orchestra.run.trial150.stdout: socat x86_64 1.7.4.1-8.el9 appstream 303 k 2026-02-21T23:05:49.011 INFO:teuthology.orchestra.run.trial150.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2026-02-21T23:05:49.011 INFO:teuthology.orchestra.run.trial150.stdout: yaml-cpp x86_64 0.6.3-5.el9 epel 122 k 2026-02-21T23:05:49.011 INFO:teuthology.orchestra.run.trial150.stdout: 2026-02-21T23:05:49.011 INFO:teuthology.orchestra.run.trial150.stdout:Transaction Summary 2026-02-21T23:05:49.011 INFO:teuthology.orchestra.run.trial150.stdout:================================================================================ 2026-02-21T23:05:49.011 INFO:teuthology.orchestra.run.trial150.stdout:Install 10 Packages 2026-02-21T23:05:49.011 INFO:teuthology.orchestra.run.trial150.stdout: 2026-02-21T23:05:49.011 INFO:teuthology.orchestra.run.trial150.stdout:Total download size: 82 M 2026-02-21T23:05:49.011 INFO:teuthology.orchestra.run.trial150.stdout:Installed size: 339 M 2026-02-21T23:05:49.011 INFO:teuthology.orchestra.run.trial150.stdout:Downloading Packages: 2026-02-21T23:05:50.075 INFO:teuthology.orchestra.run.trial150.stdout:(1/10): libcephsqlite-20.3.0-5425.g7c8cf961.el9 763 kB/s | 159 kB 00:00 2026-02-21T23:05:50.106 INFO:teuthology.orchestra.run.trial150.stdout:(2/10): hwloc-libs-2.4.1-6.el9.x86_64.rpm 8.8 MB/s | 2.1 MB 00:00 2026-02-21T23:05:50.133 INFO:teuthology.orchestra.run.trial150.stdout:(3/10): lksctp-tools-1.0.19-2.el9.x86_64.rpm 3.4 MB/s | 94 kB 00:00 2026-02-21T23:05:50.153 INFO:teuthology.orchestra.run.trial150.stdout:(4/10): libatomic-11.5.0-14.el9.x86_64.rpm 295 kB/s | 23 kB 00:00 2026-02-21T23:05:50.634 INFO:teuthology.orchestra.run.trial150.stdout:(5/10): libxslt-1.1.34-12.el9.x86_64.rpm 465 kB/s | 233 kB 00:00 2026-02-21T23:05:50.803 INFO:teuthology.orchestra.run.trial150.stdout:(6/10): socat-1.7.4.1-8.el9.x86_64.rpm 1.8 MB/s | 303 kB 00:00 2026-02-21T23:05:50.806 INFO:teuthology.orchestra.run.trial150.stdout:(7/10): protobuf-3.14.0-17.el9.x86_64.rpm 1.5 MB/s | 1.0 MB 00:00 2026-02-21T23:05:50.888 INFO:teuthology.orchestra.run.trial150.stdout:(8/10): xmlstarlet-1.6.1-20.el9.x86_64.rpm 757 kB/s | 64 kB 00:00 2026-02-21T23:05:50.955 INFO:teuthology.orchestra.run.trial150.stdout:(9/10): yaml-cpp-0.6.3-5.el9.x86_64.rpm 819 kB/s | 122 kB 00:00 2026-02-21T23:06:01.579 INFO:teuthology.orchestra.run.trial150.stdout:(10/10): ceph-test-20.3.0-5425.g7c8cf961.el9.x8 6.7 MB/s | 78 MB 00:11 2026-02-21T23:06:01.580 INFO:teuthology.orchestra.run.trial150.stdout:-------------------------------------------------------------------------------- 2026-02-21T23:06:01.580 INFO:teuthology.orchestra.run.trial150.stdout:Total 6.5 MB/s | 82 MB 00:12 2026-02-21T23:06:01.646 INFO:teuthology.orchestra.run.trial150.stdout:Running transaction check 2026-02-21T23:06:01.657 INFO:teuthology.orchestra.run.trial150.stdout:Transaction check succeeded. 2026-02-21T23:06:01.657 INFO:teuthology.orchestra.run.trial150.stdout:Running transaction test 2026-02-21T23:06:01.939 INFO:teuthology.orchestra.run.trial150.stdout:Transaction test succeeded. 2026-02-21T23:06:01.939 INFO:teuthology.orchestra.run.trial150.stdout:Running transaction 2026-02-21T23:06:02.250 INFO:teuthology.orchestra.run.trial150.stdout: Preparing : 1/1 2026-02-21T23:06:02.258 INFO:teuthology.orchestra.run.trial150.stdout: Installing : yaml-cpp-0.6.3-5.el9.x86_64 1/10 2026-02-21T23:06:02.267 INFO:teuthology.orchestra.run.trial150.stdout: Installing : socat-1.7.4.1-8.el9.x86_64 2/10 2026-02-21T23:06:02.272 INFO:teuthology.orchestra.run.trial150.stdout: Installing : protobuf-3.14.0-17.el9.x86_64 3/10 2026-02-21T23:06:02.276 INFO:teuthology.orchestra.run.trial150.stdout: Installing : libxslt-1.1.34-12.el9.x86_64 4/10 2026-02-21T23:06:02.282 INFO:teuthology.orchestra.run.trial150.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 5/10 2026-02-21T23:06:02.284 INFO:teuthology.orchestra.run.trial150.stdout: Installing : lksctp-tools-1.0.19-2.el9.x86_64 6/10 2026-02-21T23:06:02.294 INFO:teuthology.orchestra.run.trial150.stdout: Installing : libatomic-11.5.0-14.el9.x86_64 7/10 2026-02-21T23:06:02.302 INFO:teuthology.orchestra.run.trial150.stdout: Installing : hwloc-libs-2.4.1-6.el9.x86_64 8/10 2026-02-21T23:06:02.303 INFO:teuthology.orchestra.run.trial150.stdout: Installing : libcephsqlite-2:20.3.0-5425.g7c8cf961.el9.x86_64 9/10 2026-02-21T23:06:05.744 INFO:teuthology.orchestra.run.trial150.stdout: Running scriptlet: libcephsqlite-2:20.3.0-5425.g7c8cf961.el9.x86_64 9/10 2026-02-21T23:06:05.762 INFO:teuthology.orchestra.run.trial150.stdout: Installing : ceph-test-2:20.3.0-5425.g7c8cf961.el9.x86_64 10/10 2026-02-21T23:06:06.094 INFO:teuthology.orchestra.run.trial150.stdout: Running scriptlet: ceph-test-2:20.3.0-5425.g7c8cf961.el9.x86_64 10/10 2026-02-21T23:06:06.095 INFO:teuthology.orchestra.run.trial150.stdout: Verifying : ceph-test-2:20.3.0-5425.g7c8cf961.el9.x86_64 1/10 2026-02-21T23:06:06.095 INFO:teuthology.orchestra.run.trial150.stdout: Verifying : libcephsqlite-2:20.3.0-5425.g7c8cf961.el9.x86_64 2/10 2026-02-21T23:06:06.095 INFO:teuthology.orchestra.run.trial150.stdout: Verifying : hwloc-libs-2.4.1-6.el9.x86_64 3/10 2026-02-21T23:06:06.095 INFO:teuthology.orchestra.run.trial150.stdout: Verifying : libatomic-11.5.0-14.el9.x86_64 4/10 2026-02-21T23:06:06.095 INFO:teuthology.orchestra.run.trial150.stdout: Verifying : lksctp-tools-1.0.19-2.el9.x86_64 5/10 2026-02-21T23:06:06.095 INFO:teuthology.orchestra.run.trial150.stdout: Verifying : libxslt-1.1.34-12.el9.x86_64 6/10 2026-02-21T23:06:06.095 INFO:teuthology.orchestra.run.trial150.stdout: Verifying : protobuf-3.14.0-17.el9.x86_64 7/10 2026-02-21T23:06:06.095 INFO:teuthology.orchestra.run.trial150.stdout: Verifying : socat-1.7.4.1-8.el9.x86_64 8/10 2026-02-21T23:06:06.095 INFO:teuthology.orchestra.run.trial150.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 9/10 2026-02-21T23:06:06.148 INFO:teuthology.orchestra.run.trial150.stdout: Verifying : yaml-cpp-0.6.3-5.el9.x86_64 10/10 2026-02-21T23:06:06.149 INFO:teuthology.orchestra.run.trial150.stdout: 2026-02-21T23:06:06.149 INFO:teuthology.orchestra.run.trial150.stdout:Installed: 2026-02-21T23:06:06.149 INFO:teuthology.orchestra.run.trial150.stdout: ceph-test-2:20.3.0-5425.g7c8cf961.el9.x86_64 2026-02-21T23:06:06.149 INFO:teuthology.orchestra.run.trial150.stdout: hwloc-libs-2.4.1-6.el9.x86_64 2026-02-21T23:06:06.149 INFO:teuthology.orchestra.run.trial150.stdout: libatomic-11.5.0-14.el9.x86_64 2026-02-21T23:06:06.149 INFO:teuthology.orchestra.run.trial150.stdout: libcephsqlite-2:20.3.0-5425.g7c8cf961.el9.x86_64 2026-02-21T23:06:06.149 INFO:teuthology.orchestra.run.trial150.stdout: libxslt-1.1.34-12.el9.x86_64 2026-02-21T23:06:06.149 INFO:teuthology.orchestra.run.trial150.stdout: lksctp-tools-1.0.19-2.el9.x86_64 2026-02-21T23:06:06.149 INFO:teuthology.orchestra.run.trial150.stdout: protobuf-3.14.0-17.el9.x86_64 2026-02-21T23:06:06.149 INFO:teuthology.orchestra.run.trial150.stdout: socat-1.7.4.1-8.el9.x86_64 2026-02-21T23:06:06.149 INFO:teuthology.orchestra.run.trial150.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2026-02-21T23:06:06.149 INFO:teuthology.orchestra.run.trial150.stdout: yaml-cpp-0.6.3-5.el9.x86_64 2026-02-21T23:06:06.149 INFO:teuthology.orchestra.run.trial150.stdout: 2026-02-21T23:06:06.149 INFO:teuthology.orchestra.run.trial150.stdout:Complete! 2026-02-21T23:06:06.186 DEBUG:teuthology.orchestra.run.trial150:> sudo yum -y install ceph 2026-02-21T23:06:06.489 INFO:teuthology.orchestra.run.trial150.stdout:Last metadata expiration check: 0:00:40 ago on Sat 21 Feb 2026 11:05:26 PM UTC. 2026-02-21T23:06:06.574 INFO:teuthology.orchestra.run.trial150.stdout:Dependencies resolved. 2026-02-21T23:06:06.577 INFO:teuthology.orchestra.run.trial150.stdout:======================================================================================= 2026-02-21T23:06:06.577 INFO:teuthology.orchestra.run.trial150.stdout: Package Arch Version Repository Size 2026-02-21T23:06:06.577 INFO:teuthology.orchestra.run.trial150.stdout:======================================================================================= 2026-02-21T23:06:06.577 INFO:teuthology.orchestra.run.trial150.stdout:Installing: 2026-02-21T23:06:06.577 INFO:teuthology.orchestra.run.trial150.stdout: ceph x86_64 2:20.3.0-5425.g7c8cf961.el9 ceph 6.5 k 2026-02-21T23:06:06.577 INFO:teuthology.orchestra.run.trial150.stdout:Installing dependencies: 2026-02-21T23:06:06.577 INFO:teuthology.orchestra.run.trial150.stdout: abseil-cpp x86_64 20211102.0-4.el9 epel 551 k 2026-02-21T23:06:06.578 INFO:teuthology.orchestra.run.trial150.stdout: ceph-grafana-dashboards noarch 2:20.3.0-5425.g7c8cf961.el9 ceph-noarch 57 k 2026-02-21T23:06:06.578 INFO:teuthology.orchestra.run.trial150.stdout: ceph-mds x86_64 2:20.3.0-5425.g7c8cf961.el9 ceph 2.4 M 2026-02-21T23:06:06.578 INFO:teuthology.orchestra.run.trial150.stdout: ceph-mgr x86_64 2:20.3.0-5425.g7c8cf961.el9 ceph 940 k 2026-02-21T23:06:06.578 INFO:teuthology.orchestra.run.trial150.stdout: ceph-mgr-modules-core noarch 2:20.3.0-5425.g7c8cf961.el9 ceph-noarch 297 k 2026-02-21T23:06:06.578 INFO:teuthology.orchestra.run.trial150.stdout: ceph-mon x86_64 2:20.3.0-5425.g7c8cf961.el9 ceph 5.0 M 2026-02-21T23:06:06.578 INFO:teuthology.orchestra.run.trial150.stdout: ceph-osd x86_64 2:20.3.0-5425.g7c8cf961.el9 ceph 167 k 2026-02-21T23:06:06.578 INFO:teuthology.orchestra.run.trial150.stdout: ceph-osd-classic x86_64 2:20.3.0-5425.g7c8cf961.el9 ceph 16 M 2026-02-21T23:06:06.578 INFO:teuthology.orchestra.run.trial150.stdout: ceph-osd-crimson x86_64 2:20.3.0-5425.g7c8cf961.el9 ceph 19 M 2026-02-21T23:06:06.578 INFO:teuthology.orchestra.run.trial150.stdout: ceph-prometheus-alerts noarch 2:20.3.0-5425.g7c8cf961.el9 ceph-noarch 17 k 2026-02-21T23:06:06.578 INFO:teuthology.orchestra.run.trial150.stdout: cephadm noarch 2:20.3.0-5425.g7c8cf961.el9 ceph-noarch 1.0 M 2026-02-21T23:06:06.578 INFO:teuthology.orchestra.run.trial150.stdout: cryptsetup x86_64 2.8.1-3.el9 baseos 351 k 2026-02-21T23:06:06.579 INFO:teuthology.orchestra.run.trial150.stdout: flexiblas x86_64 3.0.4-9.el9 appstream 30 k 2026-02-21T23:06:06.579 INFO:teuthology.orchestra.run.trial150.stdout: flexiblas-netlib x86_64 3.0.4-9.el9 appstream 3.0 M 2026-02-21T23:06:06.579 INFO:teuthology.orchestra.run.trial150.stdout: flexiblas-openblas-openmp x86_64 3.0.4-9.el9 appstream 15 k 2026-02-21T23:06:06.579 INFO:teuthology.orchestra.run.trial150.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2026-02-21T23:06:06.579 INFO:teuthology.orchestra.run.trial150.stdout: ledmon-libs x86_64 1.1.0-3.el9 baseos 40 k 2026-02-21T23:06:06.579 INFO:teuthology.orchestra.run.trial150.stdout: libconfig x86_64 1.7.2-9.el9 baseos 72 k 2026-02-21T23:06:06.579 INFO:teuthology.orchestra.run.trial150.stdout: libgfortran x86_64 11.5.0-14.el9 baseos 794 k 2026-02-21T23:06:06.579 INFO:teuthology.orchestra.run.trial150.stdout: libquadmath x86_64 11.5.0-14.el9 baseos 184 k 2026-02-21T23:06:06.579 INFO:teuthology.orchestra.run.trial150.stdout: libstoragemgmt x86_64 1.10.1-1.el9 appstream 246 k 2026-02-21T23:06:06.579 INFO:teuthology.orchestra.run.trial150.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2026-02-21T23:06:06.579 INFO:teuthology.orchestra.run.trial150.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2026-02-21T23:06:06.579 INFO:teuthology.orchestra.run.trial150.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2026-02-21T23:06:06.579 INFO:teuthology.orchestra.run.trial150.stdout: openblas x86_64 0.3.29-1.el9 appstream 42 k 2026-02-21T23:06:06.579 INFO:teuthology.orchestra.run.trial150.stdout: openblas-openmp x86_64 0.3.29-1.el9 appstream 5.3 M 2026-02-21T23:06:06.579 INFO:teuthology.orchestra.run.trial150.stdout: protobuf-compiler x86_64 3.14.0-17.el9 crb 862 k 2026-02-21T23:06:06.579 INFO:teuthology.orchestra.run.trial150.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2026-02-21T23:06:06.579 INFO:teuthology.orchestra.run.trial150.stdout: python3-autocommand noarch 2.2.2-8.el9 epel 29 k 2026-02-21T23:06:06.579 INFO:teuthology.orchestra.run.trial150.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2026-02-21T23:06:06.579 INFO:teuthology.orchestra.run.trial150.stdout: python3-backports-tarfile noarch 1.2.0-1.el9 epel 60 k 2026-02-21T23:06:06.579 INFO:teuthology.orchestra.run.trial150.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2026-02-21T23:06:06.579 INFO:teuthology.orchestra.run.trial150.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2026-02-21T23:06:06.579 INFO:teuthology.orchestra.run.trial150.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2026-02-21T23:06:06.579 INFO:teuthology.orchestra.run.trial150.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2026-02-21T23:06:06.579 INFO:teuthology.orchestra.run.trial150.stdout: python3-cheroot noarch 10.0.1-4.el9 epel 173 k 2026-02-21T23:06:06.579 INFO:teuthology.orchestra.run.trial150.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2026-02-21T23:06:06.579 INFO:teuthology.orchestra.run.trial150.stdout: python3-cryptography x86_64 36.0.1-5.el9 baseos 1.2 M 2026-02-21T23:06:06.579 INFO:teuthology.orchestra.run.trial150.stdout: python3-devel x86_64 3.9.25-3.el9 appstream 244 k 2026-02-21T23:06:06.580 INFO:teuthology.orchestra.run.trial150.stdout: python3-google-auth noarch 1:2.45.0-1.el9 epel 254 k 2026-02-21T23:06:06.580 INFO:teuthology.orchestra.run.trial150.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2026-02-21T23:06:06.580 INFO:teuthology.orchestra.run.trial150.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2026-02-21T23:06:06.580 INFO:teuthology.orchestra.run.trial150.stdout: python3-isodate noarch 0.6.1-3.el9 epel 56 k 2026-02-21T23:06:06.580 INFO:teuthology.orchestra.run.trial150.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2026-02-21T23:06:06.580 INFO:teuthology.orchestra.run.trial150.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2026-02-21T23:06:06.580 INFO:teuthology.orchestra.run.trial150.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2026-02-21T23:06:06.580 INFO:teuthology.orchestra.run.trial150.stdout: python3-jaraco-context noarch 6.0.1-3.el9 epel 20 k 2026-02-21T23:06:06.580 INFO:teuthology.orchestra.run.trial150.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2026-02-21T23:06:06.580 INFO:teuthology.orchestra.run.trial150.stdout: python3-jaraco-text noarch 4.0.0-2.el9 epel 26 k 2026-02-21T23:06:06.580 INFO:teuthology.orchestra.run.trial150.stdout: python3-jinja2 noarch 2.11.3-8.el9 appstream 249 k 2026-02-21T23:06:06.580 INFO:teuthology.orchestra.run.trial150.stdout: python3-jmespath noarch 1.0.1-1.el9 appstream 48 k 2026-02-21T23:06:06.580 INFO:teuthology.orchestra.run.trial150.stdout: python3-kubernetes noarch 1:26.1.0-3.el9 epel 1.0 M 2026-02-21T23:06:06.580 INFO:teuthology.orchestra.run.trial150.stdout: python3-libstoragemgmt x86_64 1.10.1-1.el9 appstream 177 k 2026-02-21T23:06:06.580 INFO:teuthology.orchestra.run.trial150.stdout: python3-lxml x86_64 4.6.5-3.el9 appstream 1.2 M 2026-02-21T23:06:06.580 INFO:teuthology.orchestra.run.trial150.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2026-02-21T23:06:06.580 INFO:teuthology.orchestra.run.trial150.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2026-02-21T23:06:06.580 INFO:teuthology.orchestra.run.trial150.stdout: python3-msgpack x86_64 1.0.3-2.el9 epel 86 k 2026-02-21T23:06:06.580 INFO:teuthology.orchestra.run.trial150.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2026-02-21T23:06:06.580 INFO:teuthology.orchestra.run.trial150.stdout: python3-numpy x86_64 1:1.23.5-2.el9 appstream 6.1 M 2026-02-21T23:06:06.580 INFO:teuthology.orchestra.run.trial150.stdout: python3-numpy-f2py x86_64 1:1.23.5-2.el9 appstream 442 k 2026-02-21T23:06:06.580 INFO:teuthology.orchestra.run.trial150.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2026-02-21T23:06:06.580 INFO:teuthology.orchestra.run.trial150.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2026-02-21T23:06:06.580 INFO:teuthology.orchestra.run.trial150.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2026-02-21T23:06:06.580 INFO:teuthology.orchestra.run.trial150.stdout: python3-protobuf noarch 3.14.0-17.el9 appstream 267 k 2026-02-21T23:06:06.580 INFO:teuthology.orchestra.run.trial150.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2026-02-21T23:06:06.580 INFO:teuthology.orchestra.run.trial150.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2026-02-21T23:06:06.580 INFO:teuthology.orchestra.run.trial150.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2026-02-21T23:06:06.581 INFO:teuthology.orchestra.run.trial150.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2026-02-21T23:06:06.581 INFO:teuthology.orchestra.run.trial150.stdout: python3-pyparsing noarch 2.4.7-9.el9 baseos 150 k 2026-02-21T23:06:06.581 INFO:teuthology.orchestra.run.trial150.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2026-02-21T23:06:06.581 INFO:teuthology.orchestra.run.trial150.stdout: python3-requests noarch 2.25.1-10.el9 baseos 126 k 2026-02-21T23:06:06.581 INFO:teuthology.orchestra.run.trial150.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2026-02-21T23:06:06.581 INFO:teuthology.orchestra.run.trial150.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2026-02-21T23:06:06.581 INFO:teuthology.orchestra.run.trial150.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2026-02-21T23:06:06.581 INFO:teuthology.orchestra.run.trial150.stdout: python3-scipy x86_64 1.9.3-2.el9 appstream 19 M 2026-02-21T23:06:06.581 INFO:teuthology.orchestra.run.trial150.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2026-02-21T23:06:06.581 INFO:teuthology.orchestra.run.trial150.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2026-02-21T23:06:06.581 INFO:teuthology.orchestra.run.trial150.stdout: python3-typing-extensions noarch 4.15.0-1.el9 epel 86 k 2026-02-21T23:06:06.581 INFO:teuthology.orchestra.run.trial150.stdout: python3-urllib3 noarch 1.26.5-7.el9 baseos 218 k 2026-02-21T23:06:06.581 INFO:teuthology.orchestra.run.trial150.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2026-02-21T23:06:06.581 INFO:teuthology.orchestra.run.trial150.stdout: python3-xmlsec x86_64 1.3.13-1.el9 epel 48 k 2026-02-21T23:06:06.581 INFO:teuthology.orchestra.run.trial150.stdout: python3-xmltodict noarch 0.12.0-15.el9 epel 22 k 2026-02-21T23:06:06.582 INFO:teuthology.orchestra.run.trial150.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2026-02-21T23:06:06.582 INFO:teuthology.orchestra.run.trial150.stdout: unzip x86_64 6.0-59.el9 baseos 182 k 2026-02-21T23:06:06.582 INFO:teuthology.orchestra.run.trial150.stdout: xmlsec1 x86_64 1.2.29-13.el9 appstream 189 k 2026-02-21T23:06:06.582 INFO:teuthology.orchestra.run.trial150.stdout: xmlsec1-openssl x86_64 1.2.29-13.el9 appstream 90 k 2026-02-21T23:06:06.582 INFO:teuthology.orchestra.run.trial150.stdout: zip x86_64 3.0-35.el9 baseos 266 k 2026-02-21T23:06:06.582 INFO:teuthology.orchestra.run.trial150.stdout:Installing weak dependencies: 2026-02-21T23:06:06.582 INFO:teuthology.orchestra.run.trial150.stdout: ceph-mgr-cephadm noarch 2:20.3.0-5425.g7c8cf961.el9 ceph-noarch 184 k 2026-02-21T23:06:06.582 INFO:teuthology.orchestra.run.trial150.stdout: ceph-mgr-dashboard noarch 2:20.3.0-5425.g7c8cf961.el9 ceph-noarch 51 M 2026-02-21T23:06:06.582 INFO:teuthology.orchestra.run.trial150.stdout: ceph-mgr-diskprediction-local noarch 2:20.3.0-5425.g7c8cf961.el9 ceph-noarch 7.4 M 2026-02-21T23:06:06.582 INFO:teuthology.orchestra.run.trial150.stdout: ceph-mgr-k8sevents noarch 2:20.3.0-5425.g7c8cf961.el9 ceph-noarch 22 k 2026-02-21T23:06:06.582 INFO:teuthology.orchestra.run.trial150.stdout: ceph-mgr-rook noarch 2:20.3.0-5425.g7c8cf961.el9 ceph-noarch 50 k 2026-02-21T23:06:06.582 INFO:teuthology.orchestra.run.trial150.stdout: ceph-volume noarch 2:20.3.0-5425.g7c8cf961.el9 ceph-noarch 301 k 2026-02-21T23:06:06.582 INFO:teuthology.orchestra.run.trial150.stdout: python3-influxdb noarch 5.3.1-1.el9 epel 139 k 2026-02-21T23:06:06.582 INFO:teuthology.orchestra.run.trial150.stdout: python3-saml noarch 1.16.0-1.el9 epel 125 k 2026-02-21T23:06:06.582 INFO:teuthology.orchestra.run.trial150.stdout: 2026-02-21T23:06:06.582 INFO:teuthology.orchestra.run.trial150.stdout:Transaction Summary 2026-02-21T23:06:06.583 INFO:teuthology.orchestra.run.trial150.stdout:======================================================================================= 2026-02-21T23:06:06.583 INFO:teuthology.orchestra.run.trial150.stdout:Install 96 Packages 2026-02-21T23:06:06.583 INFO:teuthology.orchestra.run.trial150.stdout: 2026-02-21T23:06:06.583 INFO:teuthology.orchestra.run.trial150.stdout:Total download size: 158 M 2026-02-21T23:06:06.583 INFO:teuthology.orchestra.run.trial150.stdout:Installed size: 732 M 2026-02-21T23:06:06.583 INFO:teuthology.orchestra.run.trial150.stdout:Downloading Packages: 2026-02-21T23:06:07.149 INFO:teuthology.orchestra.run.trial150.stdout:(1/96): ceph-20.3.0-5425.g7c8cf961.el9.x86_64.r 100 kB/s | 6.5 kB 00:00 2026-02-21T23:06:07.261 INFO:teuthology.orchestra.run.trial150.stdout:(2/96): ceph-mds-20.3.0-5425.g7c8cf961.el9.x86_ 13 MB/s | 2.4 MB 00:00 2026-02-21T23:06:07.297 INFO:teuthology.orchestra.run.trial150.stdout:(3/96): ceph-osd-20.3.0-5425.g7c8cf961.el9.x86_ 4.6 MB/s | 167 kB 00:00 2026-02-21T23:06:07.426 INFO:teuthology.orchestra.run.trial150.stdout:(4/96): ceph-mgr-20.3.0-5425.g7c8cf961.el9.x86_ 2.7 MB/s | 940 kB 00:00 2026-02-21T23:06:08.262 INFO:teuthology.orchestra.run.trial150.stdout:(5/96): ceph-osd-crimson-20.3.0-5425.g7c8cf961. 23 MB/s | 19 MB 00:00 2026-02-21T23:06:08.382 INFO:teuthology.orchestra.run.trial150.stdout:(6/96): ceph-grafana-dashboards-20.3.0-5425.g7c 478 kB/s | 57 kB 00:00 2026-02-21T23:06:08.394 INFO:teuthology.orchestra.run.trial150.stdout:(7/96): ceph-mgr-cephadm-20.3.0-5425.g7c8cf961. 15 MB/s | 184 kB 00:00 2026-02-21T23:06:11.002 INFO:teuthology.orchestra.run.trial150.stdout:(8/96): ceph-osd-classic-20.3.0-5425.g7c8cf961. 4.2 MB/s | 16 MB 00:03 2026-02-21T23:06:11.207 INFO:teuthology.orchestra.run.trial150.stdout:(9/96): ceph-mon-20.3.0-5425.g7c8cf961.el9.x86_ 1.2 MB/s | 5.0 MB 00:04 2026-02-21T23:06:11.268 INFO:teuthology.orchestra.run.trial150.stdout:(10/96): ceph-mgr-k8sevents-20.3.0-5425.g7c8cf9 364 kB/s | 22 kB 00:00 2026-02-21T23:06:11.330 INFO:teuthology.orchestra.run.trial150.stdout:(11/96): ceph-mgr-modules-core-20.3.0-5425.g7c8 4.6 MB/s | 297 kB 00:00 2026-02-21T23:06:11.345 INFO:teuthology.orchestra.run.trial150.stdout:(12/96): ceph-mgr-diskprediction-local-20.3.0-5 22 MB/s | 7.4 MB 00:00 2026-02-21T23:06:11.380 INFO:teuthology.orchestra.run.trial150.stdout:(13/96): ceph-mgr-rook-20.3.0-5425.g7c8cf961.el 1.0 MB/s | 50 kB 00:00 2026-02-21T23:06:11.389 INFO:teuthology.orchestra.run.trial150.stdout:(14/96): ceph-prometheus-alerts-20.3.0-5425.g7c 395 kB/s | 17 kB 00:00 2026-02-21T23:06:11.425 INFO:teuthology.orchestra.run.trial150.stdout:(15/96): ceph-volume-20.3.0-5425.g7c8cf961.el9. 6.6 MB/s | 301 kB 00:00 2026-02-21T23:06:11.481 INFO:teuthology.orchestra.run.trial150.stdout:(16/96): cryptsetup-2.8.1-3.el9.x86_64.rpm 6.1 MB/s | 351 kB 00:00 2026-02-21T23:06:11.506 INFO:teuthology.orchestra.run.trial150.stdout:(17/96): ledmon-libs-1.1.0-3.el9.x86_64.rpm 1.6 MB/s | 40 kB 00:00 2026-02-21T23:06:11.509 INFO:teuthology.orchestra.run.trial150.stdout:(18/96): cephadm-20.3.0-5425.g7c8cf961.el9.noar 8.5 MB/s | 1.0 MB 00:00 2026-02-21T23:06:11.530 INFO:teuthology.orchestra.run.trial150.stdout:(19/96): libconfig-1.7.2-9.el9.x86_64.rpm 2.9 MB/s | 72 kB 00:00 2026-02-21T23:06:11.540 INFO:teuthology.orchestra.run.trial150.stdout:(20/96): libgfortran-11.5.0-14.el9.x86_64.rpm 25 MB/s | 794 kB 00:00 2026-02-21T23:06:11.557 INFO:teuthology.orchestra.run.trial150.stdout:(21/96): libquadmath-11.5.0-14.el9.x86_64.rpm 6.9 MB/s | 184 kB 00:00 2026-02-21T23:06:11.567 INFO:teuthology.orchestra.run.trial150.stdout:(22/96): python3-cffi-1.14.5-5.el9.x86_64.rpm 9.2 MB/s | 253 kB 00:00 2026-02-21T23:06:11.589 INFO:teuthology.orchestra.run.trial150.stdout:(23/96): python3-ply-3.11-14.el9.noarch.rpm 4.8 MB/s | 106 kB 00:00 2026-02-21T23:06:11.596 INFO:teuthology.orchestra.run.trial150.stdout:(24/96): python3-cryptography-36.0.1-5.el9.x86_ 32 MB/s | 1.2 MB 00:00 2026-02-21T23:06:11.614 INFO:teuthology.orchestra.run.trial150.stdout:(25/96): python3-pycparser-2.20-6.el9.noarch.rp 5.3 MB/s | 135 kB 00:00 2026-02-21T23:06:11.624 INFO:teuthology.orchestra.run.trial150.stdout:(26/96): python3-pyparsing-2.4.7-9.el9.noarch.r 5.3 MB/s | 150 kB 00:00 2026-02-21T23:06:11.636 INFO:teuthology.orchestra.run.trial150.stdout:(27/96): python3-requests-2.25.1-10.el9.noarch. 5.6 MB/s | 126 kB 00:00 2026-02-21T23:06:11.647 INFO:teuthology.orchestra.run.trial150.stdout:(28/96): python3-urllib3-1.26.5-7.el9.noarch.rp 9.2 MB/s | 218 kB 00:00 2026-02-21T23:06:11.659 INFO:teuthology.orchestra.run.trial150.stdout:(29/96): unzip-6.0-59.el9.x86_64.rpm 7.6 MB/s | 182 kB 00:00 2026-02-21T23:06:11.676 INFO:teuthology.orchestra.run.trial150.stdout:(30/96): zip-3.0-35.el9.x86_64.rpm 9.1 MB/s | 266 kB 00:00 2026-02-21T23:06:12.294 INFO:teuthology.orchestra.run.trial150.stdout:(31/96): flexiblas-3.0.4-9.el9.x86_64.rpm 47 kB/s | 30 kB 00:00 2026-02-21T23:06:12.451 INFO:teuthology.orchestra.run.trial150.stdout:(32/96): flexiblas-openblas-openmp-3.0.4-9.el9. 94 kB/s | 15 kB 00:00 2026-02-21T23:06:12.842 INFO:teuthology.orchestra.run.trial150.stdout:(33/96): libstoragemgmt-1.10.1-1.el9.x86_64.rpm 631 kB/s | 246 kB 00:00 2026-02-21T23:06:13.155 INFO:teuthology.orchestra.run.trial150.stdout:(34/96): lua-5.4.4-4.el9.x86_64.rpm 602 kB/s | 188 kB 00:00 2026-02-21T23:06:13.313 INFO:teuthology.orchestra.run.trial150.stdout:(35/96): openblas-0.3.29-1.el9.x86_64.rpm 267 kB/s | 42 kB 00:00 2026-02-21T23:06:14.491 INFO:teuthology.orchestra.run.trial150.stdout:(36/96): ceph-mgr-dashboard-20.3.0-5425.g7c8cf9 8.3 MB/s | 51 MB 00:06 2026-02-21T23:06:15.711 INFO:teuthology.orchestra.run.trial150.stdout:(37/96): flexiblas-netlib-3.0.4-9.el9.x86_64.rp 758 kB/s | 3.0 MB 00:04 2026-02-21T23:06:16.104 INFO:teuthology.orchestra.run.trial150.stdout:(38/96): python3-devel-3.9.25-3.el9.x86_64.rpm 623 kB/s | 244 kB 00:00 2026-02-21T23:06:16.496 INFO:teuthology.orchestra.run.trial150.stdout:(39/96): python3-jinja2-2.11.3-8.el9.noarch.rpm 635 kB/s | 249 kB 00:00 2026-02-21T23:06:16.655 INFO:teuthology.orchestra.run.trial150.stdout:(40/96): python3-jmespath-1.0.1-1.el9.noarch.rp 299 kB/s | 48 kB 00:00 2026-02-21T23:06:16.968 INFO:teuthology.orchestra.run.trial150.stdout:(41/96): python3-libstoragemgmt-1.10.1-1.el9.x8 563 kB/s | 177 kB 00:00 2026-02-21T23:06:18.449 INFO:teuthology.orchestra.run.trial150.stdout:(42/96): python3-lxml-4.6.5-3.el9.x86_64.rpm 846 kB/s | 1.2 MB 00:01 2026-02-21T23:06:18.608 INFO:teuthology.orchestra.run.trial150.stdout:(43/96): python3-markupsafe-1.1.1-12.el9.x86_64 218 kB/s | 35 kB 00:00 2026-02-21T23:06:19.392 INFO:teuthology.orchestra.run.trial150.stdout:(44/96): openblas-openmp-0.3.29-1.el9.x86_64.rp 890 kB/s | 5.3 MB 00:06 2026-02-21T23:06:20.018 INFO:teuthology.orchestra.run.trial150.stdout:(45/96): python3-numpy-f2py-1.23.5-2.el9.x86_64 707 kB/s | 442 kB 00:00 2026-02-21T23:06:20.254 INFO:teuthology.orchestra.run.trial150.stdout:(46/96): python3-packaging-20.9-5.el9.noarch.rp 327 kB/s | 77 kB 00:00 2026-02-21T23:06:20.571 INFO:teuthology.orchestra.run.trial150.stdout:(47/96): python3-numpy-1.23.5-2.el9.x86_64.rpm 3.1 MB/s | 6.1 MB 00:01 2026-02-21T23:06:20.649 INFO:teuthology.orchestra.run.trial150.stdout:(48/96): python3-protobuf-3.14.0-17.el9.noarch. 678 kB/s | 267 kB 00:00 2026-02-21T23:06:20.885 INFO:teuthology.orchestra.run.trial150.stdout:(49/96): python3-pyasn1-0.4.8-6.el9.noarch.rpm 505 kB/s | 159 kB 00:00 2026-02-21T23:06:21.043 INFO:teuthology.orchestra.run.trial150.stdout:(50/96): python3-requests-oauthlib-1.3.0-12.el9 338 kB/s | 54 kB 00:00 2026-02-21T23:06:22.764 INFO:teuthology.orchestra.run.trial150.stdout:(51/96): python3-babel-2.9.1-2.el9.noarch.rpm 738 kB/s | 6.0 MB 00:08 2026-02-21T23:06:23.000 INFO:teuthology.orchestra.run.trial150.stdout:(52/96): python3-toml-0.10.2-6.el9.noarch.rpm 176 kB/s | 42 kB 00:00 2026-02-21T23:06:23.313 INFO:teuthology.orchestra.run.trial150.stdout:(53/96): xmlsec1-1.2.29-13.el9.x86_64.rpm 603 kB/s | 189 kB 00:00 2026-02-21T23:06:23.472 INFO:teuthology.orchestra.run.trial150.stdout:(54/96): xmlsec1-openssl-1.2.29-13.el9.x86_64.r 568 kB/s | 90 kB 00:00 2026-02-21T23:06:23.497 INFO:teuthology.orchestra.run.trial150.stdout:(55/96): lua-devel-5.4.4-4.el9.x86_64.rpm 904 kB/s | 22 kB 00:00 2026-02-21T23:06:23.525 INFO:teuthology.orchestra.run.trial150.stdout:(56/96): protobuf-compiler-3.14.0-17.el9.x86_64 30 MB/s | 862 kB 00:00 2026-02-21T23:06:23.728 INFO:teuthology.orchestra.run.trial150.stdout:(57/96): python3-scipy-1.9.3-2.el9.x86_64.rpm 7.2 MB/s | 19 MB 00:02 2026-02-21T23:06:23.728 INFO:teuthology.orchestra.run.trial150.stdout:(58/96): abseil-cpp-20211102.0-4.el9.x86_64.rpm 2.7 MB/s | 551 kB 00:00 2026-02-21T23:06:23.755 INFO:teuthology.orchestra.run.trial150.stdout:(59/96): luarocks-3.9.2-5.el9.noarch.rpm 5.6 MB/s | 151 kB 00:00 2026-02-21T23:06:23.794 INFO:teuthology.orchestra.run.trial150.stdout:(60/96): python3-asyncssh-2.13.2-5.el9.noarch.r 14 MB/s | 548 kB 00:00 2026-02-21T23:06:23.813 INFO:teuthology.orchestra.run.trial150.stdout:(61/96): python3-pyasn1-modules-0.4.8-6.el9.noa 88 kB/s | 279 kB 00:03 2026-02-21T23:06:23.817 INFO:teuthology.orchestra.run.trial150.stdout:(62/96): grpc-data-1.46.7-10.el9.noarch.rpm 216 kB/s | 19 kB 00:00 2026-02-21T23:06:23.819 INFO:teuthology.orchestra.run.trial150.stdout:(63/96): python3-autocommand-2.2.2-8.el9.noarch 1.2 MB/s | 29 kB 00:00 2026-02-21T23:06:23.842 INFO:teuthology.orchestra.run.trial150.stdout:(64/96): python3-cachetools-4.2.4-1.el9.noarch. 1.3 MB/s | 32 kB 00:00 2026-02-21T23:06:23.867 INFO:teuthology.orchestra.run.trial150.stdout:(65/96): python3-certifi-2023.05.07-4.el9.noarc 582 kB/s | 14 kB 00:00 2026-02-21T23:06:23.881 INFO:teuthology.orchestra.run.trial150.stdout:(66/96): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 683 kB/s | 43 kB 00:00 2026-02-21T23:06:23.894 INFO:teuthology.orchestra.run.trial150.stdout:(67/96): python3-cheroot-10.0.1-4.el9.noarch.rp 6.3 MB/s | 173 kB 00:00 2026-02-21T23:06:23.923 INFO:teuthology.orchestra.run.trial150.stdout:(68/96): python3-google-auth-2.45.0-1.el9.noarc 8.7 MB/s | 254 kB 00:00 2026-02-21T23:06:23.947 INFO:teuthology.orchestra.run.trial150.stdout:(69/96): python3-backports-tarfile-1.2.0-1.el9. 448 kB/s | 60 kB 00:00 2026-02-21T23:06:23.994 INFO:teuthology.orchestra.run.trial150.stdout:(70/96): python3-cherrypy-18.6.1-2.el9.noarch.r 3.1 MB/s | 358 kB 00:00 2026-02-21T23:06:23.996 INFO:teuthology.orchestra.run.trial150.stdout:(71/96): python3-grpcio-1.46.7-10.el9.x86_64.rp 28 MB/s | 2.0 MB 00:00 2026-02-21T23:06:24.018 INFO:teuthology.orchestra.run.trial150.stdout:(72/96): python3-influxdb-5.3.1-1.el9.noarch.rp 5.7 MB/s | 139 kB 00:00 2026-02-21T23:06:24.029 INFO:teuthology.orchestra.run.trial150.stdout:(73/96): python3-isodate-0.6.1-3.el9.noarch.rpm 1.7 MB/s | 56 kB 00:00 2026-02-21T23:06:24.030 INFO:teuthology.orchestra.run.trial150.stdout:(74/96): python3-grpcio-tools-1.46.7-10.el9.x86 1.7 MB/s | 144 kB 00:00 2026-02-21T23:06:24.041 INFO:teuthology.orchestra.run.trial150.stdout:(75/96): python3-jaraco-8.2.1-3.el9.noarch.rpm 476 kB/s | 11 kB 00:00 2026-02-21T23:06:24.060 INFO:teuthology.orchestra.run.trial150.stdout:(76/96): python3-jaraco-classes-3.2.1-5.el9.noa 572 kB/s | 18 kB 00:00 2026-02-21T23:06:24.063 INFO:teuthology.orchestra.run.trial150.stdout:(77/96): python3-jaraco-collections-3.0.0-8.el9 701 kB/s | 23 kB 00:00 2026-02-21T23:06:24.063 INFO:teuthology.orchestra.run.trial150.stdout:(78/96): python3-jaraco-context-6.0.1-3.el9.noa 866 kB/s | 20 kB 00:00 2026-02-21T23:06:24.086 INFO:teuthology.orchestra.run.trial150.stdout:(79/96): python3-jaraco-text-4.0.0-2.el9.noarch 1.1 MB/s | 26 kB 00:00 2026-02-21T23:06:24.091 INFO:teuthology.orchestra.run.trial150.stdout:(80/96): python3-jaraco-functools-3.5.0-2.el9.n 626 kB/s | 19 kB 00:00 2026-02-21T23:06:24.109 INFO:teuthology.orchestra.run.trial150.stdout:(81/96): python3-more-itertools-8.12.0-2.el9.no 3.3 MB/s | 79 kB 00:00 2026-02-21T23:06:24.126 INFO:teuthology.orchestra.run.trial150.stdout:(82/96): python3-msgpack-1.0.3-2.el9.x86_64.rpm 2.5 MB/s | 86 kB 00:00 2026-02-21T23:06:24.137 INFO:teuthology.orchestra.run.trial150.stdout:(83/96): python3-natsort-7.1.1-5.el9.noarch.rpm 2.0 MB/s | 58 kB 00:00 2026-02-21T23:06:24.157 INFO:teuthology.orchestra.run.trial150.stdout:(84/96): python3-portend-3.1.0-2.el9.noarch.rpm 531 kB/s | 16 kB 00:00 2026-02-21T23:06:24.160 INFO:teuthology.orchestra.run.trial150.stdout:(85/96): python3-pyOpenSSL-21.0.0-1.el9.noarch. 3.8 MB/s | 90 kB 00:00 2026-02-21T23:06:24.185 INFO:teuthology.orchestra.run.trial150.stdout:(86/96): python3-routes-2.5.1-5.el9.noarch.rpm 7.6 MB/s | 188 kB 00:00 2026-02-21T23:06:24.186 INFO:teuthology.orchestra.run.trial150.stdout:(87/96): python3-kubernetes-26.1.0-3.el9.noarch 8.3 MB/s | 1.0 MB 00:00 2026-02-21T23:06:24.188 INFO:teuthology.orchestra.run.trial150.stdout:(88/96): python3-repoze-lru-0.7-16.el9.noarch.r 995 kB/s | 31 kB 00:00 2026-02-21T23:06:24.208 INFO:teuthology.orchestra.run.trial150.stdout:(89/96): python3-rsa-4.9-2.el9.noarch.rpm 2.5 MB/s | 59 kB 00:00 2026-02-21T23:06:24.219 INFO:teuthology.orchestra.run.trial150.stdout:(90/96): python3-tempora-5.0.0-2.el9.noarch.rpm 1.1 MB/s | 36 kB 00:00 2026-02-21T23:06:24.222 INFO:teuthology.orchestra.run.trial150.stdout:(91/96): python3-saml-1.16.0-1.el9.noarch.rpm 3.5 MB/s | 125 kB 00:00 2026-02-21T23:06:24.231 INFO:teuthology.orchestra.run.trial150.stdout:(92/96): python3-typing-extensions-4.15.0-1.el9 3.6 MB/s | 86 kB 00:00 2026-02-21T23:06:24.254 INFO:teuthology.orchestra.run.trial150.stdout:(93/96): python3-websocket-client-1.2.3-2.el9.n 2.5 MB/s | 90 kB 00:00 2026-02-21T23:06:24.254 INFO:teuthology.orchestra.run.trial150.stdout:(94/96): python3-xmltodict-0.12.0-15.el9.noarch 973 kB/s | 22 kB 00:00 2026-02-21T23:06:24.255 INFO:teuthology.orchestra.run.trial150.stdout:(95/96): python3-xmlsec-1.3.13-1.el9.x86_64.rpm 1.4 MB/s | 48 kB 00:00 2026-02-21T23:06:24.286 INFO:teuthology.orchestra.run.trial150.stdout:(96/96): python3-zc-lockfile-2.0-10.el9.noarch. 617 kB/s | 20 kB 00:00 2026-02-21T23:06:24.292 INFO:teuthology.orchestra.run.trial150.stdout:-------------------------------------------------------------------------------- 2026-02-21T23:06:24.292 INFO:teuthology.orchestra.run.trial150.stdout:Total 8.9 MB/s | 158 MB 00:17 2026-02-21T23:06:24.682 INFO:teuthology.orchestra.run.trial150.stdout:Running transaction check 2026-02-21T23:06:24.720 INFO:teuthology.orchestra.run.trial150.stdout:Transaction check succeeded. 2026-02-21T23:06:24.720 INFO:teuthology.orchestra.run.trial150.stdout:Running transaction test 2026-02-21T23:06:25.324 INFO:teuthology.orchestra.run.trial150.stdout:Transaction test succeeded. 2026-02-21T23:06:25.324 INFO:teuthology.orchestra.run.trial150.stdout:Running transaction 2026-02-21T23:06:25.969 INFO:teuthology.orchestra.run.trial150.stdout: Preparing : 1/1 2026-02-21T23:06:25.971 INFO:teuthology.orchestra.run.trial150.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/96 2026-02-21T23:06:25.976 INFO:teuthology.orchestra.run.trial150.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/96 2026-02-21T23:06:25.997 INFO:teuthology.orchestra.run.trial150.stdout: Installing : xmlsec1-1.2.29-13.el9.x86_64 3/96 2026-02-21T23:06:26.030 INFO:teuthology.orchestra.run.trial150.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 4/96 2026-02-21T23:06:26.036 INFO:teuthology.orchestra.run.trial150.stdout: Installing : python3-lxml-4.6.5-3.el9.x86_64 5/96 2026-02-21T23:06:26.039 INFO:teuthology.orchestra.run.trial150.stdout: Installing : lua-5.4.4-4.el9.x86_64 6/96 2026-02-21T23:06:26.043 INFO:teuthology.orchestra.run.trial150.stdout: Installing : flexiblas-3.0.4-9.el9.x86_64 7/96 2026-02-21T23:06:26.066 INFO:teuthology.orchestra.run.trial150.stdout: Installing : unzip-6.0-59.el9.x86_64 8/96 2026-02-21T23:06:26.079 INFO:teuthology.orchestra.run.trial150.stdout: Installing : python3-urllib3-1.26.5-7.el9.noarch 9/96 2026-02-21T23:06:26.082 INFO:teuthology.orchestra.run.trial150.stdout: Installing : python3-requests-2.25.1-10.el9.noarch 10/96 2026-02-21T23:06:26.088 INFO:teuthology.orchestra.run.trial150.stdout: Installing : libquadmath-11.5.0-14.el9.x86_64 11/96 2026-02-21T23:06:26.089 INFO:teuthology.orchestra.run.trial150.stdout: Installing : libgfortran-11.5.0-14.el9.x86_64 12/96 2026-02-21T23:06:26.100 INFO:teuthology.orchestra.run.trial150.stdout: Installing : ledmon-libs-1.1.0-3.el9.x86_64 13/96 2026-02-21T23:06:26.106 INFO:teuthology.orchestra.run.trial150.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 14/96 2026-02-21T23:06:26.130 INFO:teuthology.orchestra.run.trial150.stdout: Installing : zip-3.0-35.el9.x86_64 15/96 2026-02-21T23:06:26.133 INFO:teuthology.orchestra.run.trial150.stdout: Installing : luarocks-3.9.2-5.el9.noarch 16/96 2026-02-21T23:06:26.183 INFO:teuthology.orchestra.run.trial150.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 17/96 2026-02-21T23:06:26.196 INFO:teuthology.orchestra.run.trial150.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 18/96 2026-02-21T23:06:26.198 INFO:teuthology.orchestra.run.trial150.stdout: Installing : python3-rsa-4.9-2.el9.noarch 19/96 2026-02-21T23:06:26.202 INFO:teuthology.orchestra.run.trial150.stdout: Installing : xmlsec1-openssl-1.2.29-13.el9.x86_64 20/96 2026-02-21T23:06:26.208 INFO:teuthology.orchestra.run.trial150.stdout: Installing : python3-xmlsec-1.3.13-1.el9.x86_64 21/96 2026-02-21T23:06:26.212 INFO:teuthology.orchestra.run.trial150.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 22/96 2026-02-21T23:06:26.215 INFO:teuthology.orchestra.run.trial150.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 23/96 2026-02-21T23:06:26.230 INFO:teuthology.orchestra.run.trial150.stdout: Installing : python3-xmltodict-0.12.0-15.el9.noarch 24/96 2026-02-21T23:06:26.234 INFO:teuthology.orchestra.run.trial150.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 25/96 2026-02-21T23:06:26.239 INFO:teuthology.orchestra.run.trial150.stdout: Installing : python3-typing-extensions-4.15.0-1.el9.noarch 26/96 2026-02-21T23:06:26.250 INFO:teuthology.orchestra.run.trial150.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 27/96 2026-02-21T23:06:26.259 INFO:teuthology.orchestra.run.trial150.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 28/96 2026-02-21T23:06:26.266 INFO:teuthology.orchestra.run.trial150.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 29/96 2026-02-21T23:06:26.285 INFO:teuthology.orchestra.run.trial150.stdout: Installing : python3-msgpack-1.0.3-2.el9.x86_64 30/96 2026-02-21T23:06:26.296 INFO:teuthology.orchestra.run.trial150.stdout: Installing : python3-influxdb-5.3.1-1.el9.noarch 31/96 2026-02-21T23:06:26.311 INFO:teuthology.orchestra.run.trial150.stdout: Installing : python3-isodate-0.6.1-3.el9.noarch 32/96 2026-02-21T23:06:26.316 INFO:teuthology.orchestra.run.trial150.stdout: Installing : python3-saml-1.16.0-1.el9.noarch 33/96 2026-02-21T23:06:26.324 INFO:teuthology.orchestra.run.trial150.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 34/96 2026-02-21T23:06:26.363 INFO:teuthology.orchestra.run.trial150.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 35/96 2026-02-21T23:06:26.683 INFO:teuthology.orchestra.run.trial150.stdout: Installing : python3-google-auth-1:2.45.0-1.el9.noarch 36/96 2026-02-21T23:06:26.693 INFO:teuthology.orchestra.run.trial150.stdout: Installing : python3-kubernetes-1:26.1.0-3.el9.noarch 37/96 2026-02-21T23:06:26.697 INFO:teuthology.orchestra.run.trial150.stdout: Installing : python3-backports-tarfile-1.2.0-1.el9.noarch 38/96 2026-02-21T23:06:26.702 INFO:teuthology.orchestra.run.trial150.stdout: Installing : python3-jaraco-context-6.0.1-3.el9.noarch 39/96 2026-02-21T23:06:26.704 INFO:teuthology.orchestra.run.trial150.stdout: Installing : python3-autocommand-2.2.2-8.el9.noarch 40/96 2026-02-21T23:06:26.732 INFO:teuthology.orchestra.run.trial150.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 41/96 2026-02-21T23:06:26.775 INFO:teuthology.orchestra.run.trial150.stdout: Installing : abseil-cpp-20211102.0-4.el9.x86_64 42/96 2026-02-21T23:06:26.783 INFO:teuthology.orchestra.run.trial150.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 43/96 2026-02-21T23:06:26.789 INFO:teuthology.orchestra.run.trial150.stdout: Installing : protobuf-compiler-3.14.0-17.el9.x86_64 44/96 2026-02-21T23:06:26.792 INFO:teuthology.orchestra.run.trial150.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 45/96 2026-02-21T23:06:26.798 INFO:teuthology.orchestra.run.trial150.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 46/96 2026-02-21T23:06:26.801 INFO:teuthology.orchestra.run.trial150.stdout: Installing : python3-jaraco-text-4.0.0-2.el9.noarch 47/96 2026-02-21T23:06:26.808 INFO:teuthology.orchestra.run.trial150.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 48/96 2026-02-21T23:06:26.812 INFO:teuthology.orchestra.run.trial150.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 49/96 2026-02-21T23:06:26.839 INFO:teuthology.orchestra.run.trial150.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 50/96 2026-02-21T23:06:26.849 INFO:teuthology.orchestra.run.trial150.stdout: Installing : python3-protobuf-3.14.0-17.el9.noarch 51/96 2026-02-21T23:06:26.855 INFO:teuthology.orchestra.run.trial150.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 52/96 2026-02-21T23:06:26.861 INFO:teuthology.orchestra.run.trial150.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 53/96 2026-02-21T23:06:26.896 INFO:teuthology.orchestra.run.trial150.stdout: Installing : python3-jmespath-1.0.1-1.el9.noarch 54/96 2026-02-21T23:06:27.118 INFO:teuthology.orchestra.run.trial150.stdout: Installing : python3-devel-3.9.25-3.el9.x86_64 55/96 2026-02-21T23:06:27.141 INFO:teuthology.orchestra.run.trial150.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 56/96 2026-02-21T23:06:27.145 INFO:teuthology.orchestra.run.trial150.stdout: Installing : python3-jinja2-2.11.3-8.el9.noarch 57/96 2026-02-21T23:06:27.204 INFO:teuthology.orchestra.run.trial150.stdout: Installing : openblas-0.3.29-1.el9.x86_64 58/96 2026-02-21T23:06:27.206 INFO:teuthology.orchestra.run.trial150.stdout: Installing : openblas-openmp-0.3.29-1.el9.x86_64 59/96 2026-02-21T23:06:27.227 INFO:teuthology.orchestra.run.trial150.stdout: Installing : flexiblas-openblas-openmp-3.0.4-9.el9.x86_64 60/96 2026-02-21T23:06:27.550 INFO:teuthology.orchestra.run.trial150.stdout: Installing : flexiblas-netlib-3.0.4-9.el9.x86_64 61/96 2026-02-21T23:06:27.622 INFO:teuthology.orchestra.run.trial150.stdout: Installing : python3-numpy-1:1.23.5-2.el9.x86_64 62/96 2026-02-21T23:06:28.289 INFO:teuthology.orchestra.run.trial150.stdout: Installing : python3-numpy-f2py-1:1.23.5-2.el9.x86_64 63/96 2026-02-21T23:06:28.305 INFO:teuthology.orchestra.run.trial150.stdout: Installing : python3-scipy-1.9.3-2.el9.x86_64 64/96 2026-02-21T23:06:28.315 INFO:teuthology.orchestra.run.trial150.stdout: Installing : python3-pyparsing-2.4.7-9.el9.noarch 65/96 2026-02-21T23:06:28.322 INFO:teuthology.orchestra.run.trial150.stdout: Installing : python3-packaging-20.9-5.el9.noarch 66/96 2026-02-21T23:06:28.335 INFO:teuthology.orchestra.run.trial150.stdout: Installing : python3-ply-3.11-14.el9.noarch 67/96 2026-02-21T23:06:28.350 INFO:teuthology.orchestra.run.trial150.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 68/96 2026-02-21T23:06:28.425 INFO:teuthology.orchestra.run.trial150.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 69/96 2026-02-21T23:06:28.434 INFO:teuthology.orchestra.run.trial150.stdout: Installing : python3-cryptography-36.0.1-5.el9.x86_64 70/96 2026-02-21T23:06:28.457 INFO:teuthology.orchestra.run.trial150.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 71/96 2026-02-21T23:06:28.487 INFO:teuthology.orchestra.run.trial150.stdout: Installing : python3-cheroot-10.0.1-4.el9.noarch 72/96 2026-02-21T23:06:28.536 INFO:teuthology.orchestra.run.trial150.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 73/96 2026-02-21T23:06:28.542 INFO:teuthology.orchestra.run.trial150.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 74/96 2026-02-21T23:06:28.548 INFO:teuthology.orchestra.run.trial150.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 75/96 2026-02-21T23:06:28.549 INFO:teuthology.orchestra.run.trial150.stdout: Installing : libconfig-1.7.2-9.el9.x86_64 76/96 2026-02-21T23:06:28.567 INFO:teuthology.orchestra.run.trial150.stdout: Running scriptlet: libstoragemgmt-1.10.1-1.el9.x86_64 77/96 2026-02-21T23:06:28.567 INFO:teuthology.orchestra.run.trial150.stdout:Creating group 'libstoragemgmt' with GID 991. 2026-02-21T23:06:28.567 INFO:teuthology.orchestra.run.trial150.stdout:Creating user 'libstoragemgmt' (daemon account for libstoragemgmt) with UID 991 and GID 991. 2026-02-21T23:06:28.567 INFO:teuthology.orchestra.run.trial150.stdout: 2026-02-21T23:06:28.574 INFO:teuthology.orchestra.run.trial150.stdout: Installing : libstoragemgmt-1.10.1-1.el9.x86_64 77/96 2026-02-21T23:06:28.599 INFO:teuthology.orchestra.run.trial150.stdout: Running scriptlet: libstoragemgmt-1.10.1-1.el9.x86_64 77/96 2026-02-21T23:06:28.599 INFO:teuthology.orchestra.run.trial150.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/libstoragemgmt.service → /usr/lib/systemd/system/libstoragemgmt.service. 2026-02-21T23:06:28.600 INFO:teuthology.orchestra.run.trial150.stdout: 2026-02-21T23:06:28.632 INFO:teuthology.orchestra.run.trial150.stdout: Installing : python3-libstoragemgmt-1.10.1-1.el9.x86_64 78/96 2026-02-21T23:06:29.268 INFO:teuthology.orchestra.run.trial150.stdout: Installing : cryptsetup-2.8.1-3.el9.x86_64 79/96 2026-02-21T23:06:30.047 INFO:teuthology.orchestra.run.trial150.stdout: Installing : ceph-osd-classic-2:20.3.0-5425.g7c8cf961.el9.x86_6 80/96 2026-02-21T23:06:30.111 INFO:teuthology.orchestra.run.trial150.stdout: Installing : ceph-osd-crimson-2:20.3.0-5425.g7c8cf961.el9.x86_6 81/96 2026-02-21T23:06:30.113 INFO:teuthology.orchestra.run.trial150.stdout: Installing : ceph-volume-2:20.3.0-5425.g7c8cf961.el9.noarch 82/96 2026-02-21T23:06:30.127 INFO:teuthology.orchestra.run.trial150.stdout: Running scriptlet: ceph-volume-2:20.3.0-5425.g7c8cf961.el9.noarch 82/96 2026-02-21T23:06:30.127 INFO:teuthology.orchestra.run.trial150.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-21T23:06:30.127 INFO:teuthology.orchestra.run.trial150.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2026-02-21T23:06:30.127 INFO:teuthology.orchestra.run.trial150.stdout: 2026-02-21T23:06:30.139 INFO:teuthology.orchestra.run.trial150.stdout: Installing : ceph-osd-2:20.3.0-5425.g7c8cf961.el9.x86_64 83/96 2026-02-21T23:06:30.163 INFO:teuthology.orchestra.run.trial150.stdout: Running scriptlet: ceph-osd-2:20.3.0-5425.g7c8cf961.el9.x86_64 83/96 2026-02-21T23:06:30.164 INFO:teuthology.orchestra.run.trial150.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-21T23:06:30.164 INFO:teuthology.orchestra.run.trial150.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2026-02-21T23:06:30.164 INFO:teuthology.orchestra.run.trial150.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2026-02-21T23:06:30.164 INFO:teuthology.orchestra.run.trial150.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2026-02-21T23:06:30.164 INFO:teuthology.orchestra.run.trial150.stdout: 2026-02-21T23:06:30.220 INFO:teuthology.orchestra.run.trial150.stdout: Running scriptlet: cephadm-2:20.3.0-5425.g7c8cf961.el9.noarch 84/96 2026-02-21T23:06:30.221 INFO:teuthology.orchestra.run.trial150.stdout: Installing : cephadm-2:20.3.0-5425.g7c8cf961.el9.noarch 84/96 2026-02-21T23:06:30.230 INFO:teuthology.orchestra.run.trial150.stdout: Installing : ceph-prometheus-alerts-2:20.3.0-5425.g7c8cf961.el9 85/96 2026-02-21T23:06:30.255 INFO:teuthology.orchestra.run.trial150.stdout: Installing : ceph-grafana-dashboards-2:20.3.0-5425.g7c8cf961.el 86/96 2026-02-21T23:06:30.256 INFO:teuthology.orchestra.run.trial150.stdout: Installing : ceph-mgr-cephadm-2:20.3.0-5425.g7c8cf961.el9.noarc 87/96 2026-02-21T23:06:33.186 INFO:teuthology.orchestra.run.trial150.stdout: Running scriptlet: ceph-mgr-cephadm-2:20.3.0-5425.g7c8cf961.el9.noarc 87/96 2026-02-21T23:06:33.193 INFO:teuthology.orchestra.run.trial150.stdout: Installing : ceph-mgr-dashboard-2:20.3.0-5425.g7c8cf961.el9.noa 88/96 2026-02-21T23:06:33.655 INFO:teuthology.orchestra.run.trial150.stdout: Running scriptlet: ceph-mgr-dashboard-2:20.3.0-5425.g7c8cf961.el9.noa 88/96 2026-02-21T23:06:33.656 INFO:teuthology.orchestra.run.trial150.stdout: Installing : ceph-mgr-diskprediction-local-2:20.3.0-5425.g7c8cf 89/96 2026-02-21T23:06:33.669 INFO:teuthology.orchestra.run.trial150.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:20.3.0-5425.g7c8cf 89/96 2026-02-21T23:06:33.670 INFO:teuthology.orchestra.run.trial150.stdout: Installing : ceph-mgr-k8sevents-2:20.3.0-5425.g7c8cf961.el9.noa 90/96 2026-02-21T23:06:33.733 INFO:teuthology.orchestra.run.trial150.stdout: Running scriptlet: ceph-mgr-k8sevents-2:20.3.0-5425.g7c8cf961.el9.noa 90/96 2026-02-21T23:06:33.778 INFO:teuthology.orchestra.run.trial150.stdout: Installing : ceph-mgr-modules-core-2:20.3.0-5425.g7c8cf961.el9. 91/96 2026-02-21T23:06:33.779 INFO:teuthology.orchestra.run.trial150.stdout: Installing : ceph-mgr-2:20.3.0-5425.g7c8cf961.el9.x86_64 92/96 2026-02-21T23:06:33.800 INFO:teuthology.orchestra.run.trial150.stdout: Running scriptlet: ceph-mgr-2:20.3.0-5425.g7c8cf961.el9.x86_64 92/96 2026-02-21T23:06:33.800 INFO:teuthology.orchestra.run.trial150.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-21T23:06:33.801 INFO:teuthology.orchestra.run.trial150.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2026-02-21T23:06:33.801 INFO:teuthology.orchestra.run.trial150.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2026-02-21T23:06:33.801 INFO:teuthology.orchestra.run.trial150.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2026-02-21T23:06:33.801 INFO:teuthology.orchestra.run.trial150.stdout: 2026-02-21T23:06:33.812 INFO:teuthology.orchestra.run.trial150.stdout: Installing : ceph-mgr-rook-2:20.3.0-5425.g7c8cf961.el9.noarch 93/96 2026-02-21T23:06:34.035 INFO:teuthology.orchestra.run.trial150.stdout: Running scriptlet: ceph-mgr-rook-2:20.3.0-5425.g7c8cf961.el9.noarch 93/96 2026-02-21T23:06:34.036 INFO:teuthology.orchestra.run.trial150.stdout: Installing : ceph-mon-2:20.3.0-5425.g7c8cf961.el9.x86_64 94/96 2026-02-21T23:06:34.057 INFO:teuthology.orchestra.run.trial150.stdout: Running scriptlet: ceph-mon-2:20.3.0-5425.g7c8cf961.el9.x86_64 94/96 2026-02-21T23:06:34.058 INFO:teuthology.orchestra.run.trial150.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-21T23:06:34.058 INFO:teuthology.orchestra.run.trial150.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2026-02-21T23:06:34.058 INFO:teuthology.orchestra.run.trial150.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2026-02-21T23:06:34.058 INFO:teuthology.orchestra.run.trial150.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2026-02-21T23:06:34.058 INFO:teuthology.orchestra.run.trial150.stdout: 2026-02-21T23:06:34.154 INFO:teuthology.orchestra.run.trial150.stdout: Installing : ceph-mds-2:20.3.0-5425.g7c8cf961.el9.x86_64 95/96 2026-02-21T23:06:34.175 INFO:teuthology.orchestra.run.trial150.stdout: Running scriptlet: ceph-mds-2:20.3.0-5425.g7c8cf961.el9.x86_64 95/96 2026-02-21T23:06:34.175 INFO:teuthology.orchestra.run.trial150.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-21T23:06:34.175 INFO:teuthology.orchestra.run.trial150.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2026-02-21T23:06:34.175 INFO:teuthology.orchestra.run.trial150.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2026-02-21T23:06:34.175 INFO:teuthology.orchestra.run.trial150.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2026-02-21T23:06:34.175 INFO:teuthology.orchestra.run.trial150.stdout: 2026-02-21T23:06:34.176 INFO:teuthology.orchestra.run.trial150.stdout: Installing : ceph-2:20.3.0-5425.g7c8cf961.el9.x86_64 96/96 2026-02-21T23:06:34.182 INFO:teuthology.orchestra.run.trial150.stdout: Running scriptlet: ceph-osd-classic-2:20.3.0-5425.g7c8cf961.el9.x86_6 96/96 2026-02-21T23:06:34.205 INFO:teuthology.orchestra.run.trial150.stdout: Running scriptlet: ceph-osd-crimson-2:20.3.0-5425.g7c8cf961.el9.x86_6 96/96 2026-02-21T23:06:35.043 INFO:teuthology.orchestra.run.trial150.stdout: Running scriptlet: ceph-2:20.3.0-5425.g7c8cf961.el9.x86_64 96/96 2026-02-21T23:06:35.043 INFO:teuthology.orchestra.run.trial150.stdout: Verifying : ceph-2:20.3.0-5425.g7c8cf961.el9.x86_64 1/96 2026-02-21T23:06:35.043 INFO:teuthology.orchestra.run.trial150.stdout: Verifying : ceph-mds-2:20.3.0-5425.g7c8cf961.el9.x86_64 2/96 2026-02-21T23:06:35.043 INFO:teuthology.orchestra.run.trial150.stdout: Verifying : ceph-mgr-2:20.3.0-5425.g7c8cf961.el9.x86_64 3/96 2026-02-21T23:06:35.044 INFO:teuthology.orchestra.run.trial150.stdout: Verifying : ceph-mon-2:20.3.0-5425.g7c8cf961.el9.x86_64 4/96 2026-02-21T23:06:35.044 INFO:teuthology.orchestra.run.trial150.stdout: Verifying : ceph-osd-2:20.3.0-5425.g7c8cf961.el9.x86_64 5/96 2026-02-21T23:06:35.044 INFO:teuthology.orchestra.run.trial150.stdout: Verifying : ceph-osd-classic-2:20.3.0-5425.g7c8cf961.el9.x86_6 6/96 2026-02-21T23:06:35.044 INFO:teuthology.orchestra.run.trial150.stdout: Verifying : ceph-osd-crimson-2:20.3.0-5425.g7c8cf961.el9.x86_6 7/96 2026-02-21T23:06:35.044 INFO:teuthology.orchestra.run.trial150.stdout: Verifying : ceph-grafana-dashboards-2:20.3.0-5425.g7c8cf961.el 8/96 2026-02-21T23:06:35.044 INFO:teuthology.orchestra.run.trial150.stdout: Verifying : ceph-mgr-cephadm-2:20.3.0-5425.g7c8cf961.el9.noarc 9/96 2026-02-21T23:06:35.044 INFO:teuthology.orchestra.run.trial150.stdout: Verifying : ceph-mgr-dashboard-2:20.3.0-5425.g7c8cf961.el9.noa 10/96 2026-02-21T23:06:35.044 INFO:teuthology.orchestra.run.trial150.stdout: Verifying : ceph-mgr-diskprediction-local-2:20.3.0-5425.g7c8cf 11/96 2026-02-21T23:06:35.044 INFO:teuthology.orchestra.run.trial150.stdout: Verifying : ceph-mgr-k8sevents-2:20.3.0-5425.g7c8cf961.el9.noa 12/96 2026-02-21T23:06:35.044 INFO:teuthology.orchestra.run.trial150.stdout: Verifying : ceph-mgr-modules-core-2:20.3.0-5425.g7c8cf961.el9. 13/96 2026-02-21T23:06:35.044 INFO:teuthology.orchestra.run.trial150.stdout: Verifying : ceph-mgr-rook-2:20.3.0-5425.g7c8cf961.el9.noarch 14/96 2026-02-21T23:06:35.044 INFO:teuthology.orchestra.run.trial150.stdout: Verifying : ceph-prometheus-alerts-2:20.3.0-5425.g7c8cf961.el9 15/96 2026-02-21T23:06:35.045 INFO:teuthology.orchestra.run.trial150.stdout: Verifying : ceph-volume-2:20.3.0-5425.g7c8cf961.el9.noarch 16/96 2026-02-21T23:06:35.045 INFO:teuthology.orchestra.run.trial150.stdout: Verifying : cephadm-2:20.3.0-5425.g7c8cf961.el9.noarch 17/96 2026-02-21T23:06:35.045 INFO:teuthology.orchestra.run.trial150.stdout: Verifying : cryptsetup-2.8.1-3.el9.x86_64 18/96 2026-02-21T23:06:35.045 INFO:teuthology.orchestra.run.trial150.stdout: Verifying : ledmon-libs-1.1.0-3.el9.x86_64 19/96 2026-02-21T23:06:35.045 INFO:teuthology.orchestra.run.trial150.stdout: Verifying : libconfig-1.7.2-9.el9.x86_64 20/96 2026-02-21T23:06:35.045 INFO:teuthology.orchestra.run.trial150.stdout: Verifying : libgfortran-11.5.0-14.el9.x86_64 21/96 2026-02-21T23:06:35.045 INFO:teuthology.orchestra.run.trial150.stdout: Verifying : libquadmath-11.5.0-14.el9.x86_64 22/96 2026-02-21T23:06:35.045 INFO:teuthology.orchestra.run.trial150.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 23/96 2026-02-21T23:06:35.045 INFO:teuthology.orchestra.run.trial150.stdout: Verifying : python3-cryptography-36.0.1-5.el9.x86_64 24/96 2026-02-21T23:06:35.045 INFO:teuthology.orchestra.run.trial150.stdout: Verifying : python3-ply-3.11-14.el9.noarch 25/96 2026-02-21T23:06:35.045 INFO:teuthology.orchestra.run.trial150.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 26/96 2026-02-21T23:06:35.045 INFO:teuthology.orchestra.run.trial150.stdout: Verifying : python3-pyparsing-2.4.7-9.el9.noarch 27/96 2026-02-21T23:06:35.046 INFO:teuthology.orchestra.run.trial150.stdout: Verifying : python3-requests-2.25.1-10.el9.noarch 28/96 2026-02-21T23:06:35.046 INFO:teuthology.orchestra.run.trial150.stdout: Verifying : python3-urllib3-1.26.5-7.el9.noarch 29/96 2026-02-21T23:06:35.046 INFO:teuthology.orchestra.run.trial150.stdout: Verifying : unzip-6.0-59.el9.x86_64 30/96 2026-02-21T23:06:35.046 INFO:teuthology.orchestra.run.trial150.stdout: Verifying : zip-3.0-35.el9.x86_64 31/96 2026-02-21T23:06:35.046 INFO:teuthology.orchestra.run.trial150.stdout: Verifying : flexiblas-3.0.4-9.el9.x86_64 32/96 2026-02-21T23:06:35.046 INFO:teuthology.orchestra.run.trial150.stdout: Verifying : flexiblas-netlib-3.0.4-9.el9.x86_64 33/96 2026-02-21T23:06:35.046 INFO:teuthology.orchestra.run.trial150.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-9.el9.x86_64 34/96 2026-02-21T23:06:35.046 INFO:teuthology.orchestra.run.trial150.stdout: Verifying : libstoragemgmt-1.10.1-1.el9.x86_64 35/96 2026-02-21T23:06:35.046 INFO:teuthology.orchestra.run.trial150.stdout: Verifying : lua-5.4.4-4.el9.x86_64 36/96 2026-02-21T23:06:35.046 INFO:teuthology.orchestra.run.trial150.stdout: Verifying : openblas-0.3.29-1.el9.x86_64 37/96 2026-02-21T23:06:35.046 INFO:teuthology.orchestra.run.trial150.stdout: Verifying : openblas-openmp-0.3.29-1.el9.x86_64 38/96 2026-02-21T23:06:35.046 INFO:teuthology.orchestra.run.trial150.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 39/96 2026-02-21T23:06:35.047 INFO:teuthology.orchestra.run.trial150.stdout: Verifying : python3-devel-3.9.25-3.el9.x86_64 40/96 2026-02-21T23:06:35.047 INFO:teuthology.orchestra.run.trial150.stdout: Verifying : python3-jinja2-2.11.3-8.el9.noarch 41/96 2026-02-21T23:06:35.047 INFO:teuthology.orchestra.run.trial150.stdout: Verifying : python3-jmespath-1.0.1-1.el9.noarch 42/96 2026-02-21T23:06:35.047 INFO:teuthology.orchestra.run.trial150.stdout: Verifying : python3-libstoragemgmt-1.10.1-1.el9.x86_64 43/96 2026-02-21T23:06:35.047 INFO:teuthology.orchestra.run.trial150.stdout: Verifying : python3-lxml-4.6.5-3.el9.x86_64 44/96 2026-02-21T23:06:35.047 INFO:teuthology.orchestra.run.trial150.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 45/96 2026-02-21T23:06:35.047 INFO:teuthology.orchestra.run.trial150.stdout: Verifying : python3-numpy-1:1.23.5-2.el9.x86_64 46/96 2026-02-21T23:06:35.047 INFO:teuthology.orchestra.run.trial150.stdout: Verifying : python3-numpy-f2py-1:1.23.5-2.el9.x86_64 47/96 2026-02-21T23:06:35.047 INFO:teuthology.orchestra.run.trial150.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 48/96 2026-02-21T23:06:35.047 INFO:teuthology.orchestra.run.trial150.stdout: Verifying : python3-protobuf-3.14.0-17.el9.noarch 49/96 2026-02-21T23:06:35.047 INFO:teuthology.orchestra.run.trial150.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 50/96 2026-02-21T23:06:35.047 INFO:teuthology.orchestra.run.trial150.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 51/96 2026-02-21T23:06:35.047 INFO:teuthology.orchestra.run.trial150.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 52/96 2026-02-21T23:06:35.048 INFO:teuthology.orchestra.run.trial150.stdout: Verifying : python3-scipy-1.9.3-2.el9.x86_64 53/96 2026-02-21T23:06:35.048 INFO:teuthology.orchestra.run.trial150.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 54/96 2026-02-21T23:06:35.048 INFO:teuthology.orchestra.run.trial150.stdout: Verifying : xmlsec1-1.2.29-13.el9.x86_64 55/96 2026-02-21T23:06:35.048 INFO:teuthology.orchestra.run.trial150.stdout: Verifying : xmlsec1-openssl-1.2.29-13.el9.x86_64 56/96 2026-02-21T23:06:35.048 INFO:teuthology.orchestra.run.trial150.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 57/96 2026-02-21T23:06:35.048 INFO:teuthology.orchestra.run.trial150.stdout: Verifying : protobuf-compiler-3.14.0-17.el9.x86_64 58/96 2026-02-21T23:06:35.048 INFO:teuthology.orchestra.run.trial150.stdout: Verifying : abseil-cpp-20211102.0-4.el9.x86_64 59/96 2026-02-21T23:06:35.048 INFO:teuthology.orchestra.run.trial150.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 60/96 2026-02-21T23:06:35.048 INFO:teuthology.orchestra.run.trial150.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 61/96 2026-02-21T23:06:35.048 INFO:teuthology.orchestra.run.trial150.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 62/96 2026-02-21T23:06:35.048 INFO:teuthology.orchestra.run.trial150.stdout: Verifying : python3-autocommand-2.2.2-8.el9.noarch 63/96 2026-02-21T23:06:35.048 INFO:teuthology.orchestra.run.trial150.stdout: Verifying : python3-backports-tarfile-1.2.0-1.el9.noarch 64/96 2026-02-21T23:06:35.049 INFO:teuthology.orchestra.run.trial150.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 65/96 2026-02-21T23:06:35.049 INFO:teuthology.orchestra.run.trial150.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 66/96 2026-02-21T23:06:35.049 INFO:teuthology.orchestra.run.trial150.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 67/96 2026-02-21T23:06:35.049 INFO:teuthology.orchestra.run.trial150.stdout: Verifying : python3-cheroot-10.0.1-4.el9.noarch 68/96 2026-02-21T23:06:35.049 INFO:teuthology.orchestra.run.trial150.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 69/96 2026-02-21T23:06:35.049 INFO:teuthology.orchestra.run.trial150.stdout: Verifying : python3-google-auth-1:2.45.0-1.el9.noarch 70/96 2026-02-21T23:06:35.049 INFO:teuthology.orchestra.run.trial150.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 71/96 2026-02-21T23:06:35.049 INFO:teuthology.orchestra.run.trial150.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 72/96 2026-02-21T23:06:35.049 INFO:teuthology.orchestra.run.trial150.stdout: Verifying : python3-influxdb-5.3.1-1.el9.noarch 73/96 2026-02-21T23:06:35.049 INFO:teuthology.orchestra.run.trial150.stdout: Verifying : python3-isodate-0.6.1-3.el9.noarch 74/96 2026-02-21T23:06:35.049 INFO:teuthology.orchestra.run.trial150.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 75/96 2026-02-21T23:06:35.049 INFO:teuthology.orchestra.run.trial150.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 76/96 2026-02-21T23:06:35.049 INFO:teuthology.orchestra.run.trial150.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 77/96 2026-02-21T23:06:35.050 INFO:teuthology.orchestra.run.trial150.stdout: Verifying : python3-jaraco-context-6.0.1-3.el9.noarch 78/96 2026-02-21T23:06:35.050 INFO:teuthology.orchestra.run.trial150.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 79/96 2026-02-21T23:06:35.050 INFO:teuthology.orchestra.run.trial150.stdout: Verifying : python3-jaraco-text-4.0.0-2.el9.noarch 80/96 2026-02-21T23:06:35.050 INFO:teuthology.orchestra.run.trial150.stdout: Verifying : python3-kubernetes-1:26.1.0-3.el9.noarch 81/96 2026-02-21T23:06:35.050 INFO:teuthology.orchestra.run.trial150.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 82/96 2026-02-21T23:06:35.050 INFO:teuthology.orchestra.run.trial150.stdout: Verifying : python3-msgpack-1.0.3-2.el9.x86_64 83/96 2026-02-21T23:06:35.050 INFO:teuthology.orchestra.run.trial150.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 84/96 2026-02-21T23:06:35.050 INFO:teuthology.orchestra.run.trial150.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 85/96 2026-02-21T23:06:35.050 INFO:teuthology.orchestra.run.trial150.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 86/96 2026-02-21T23:06:35.050 INFO:teuthology.orchestra.run.trial150.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 87/96 2026-02-21T23:06:35.050 INFO:teuthology.orchestra.run.trial150.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 88/96 2026-02-21T23:06:35.050 INFO:teuthology.orchestra.run.trial150.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 89/96 2026-02-21T23:06:35.050 INFO:teuthology.orchestra.run.trial150.stdout: Verifying : python3-saml-1.16.0-1.el9.noarch 90/96 2026-02-21T23:06:35.051 INFO:teuthology.orchestra.run.trial150.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 91/96 2026-02-21T23:06:35.051 INFO:teuthology.orchestra.run.trial150.stdout: Verifying : python3-typing-extensions-4.15.0-1.el9.noarch 92/96 2026-02-21T23:06:35.051 INFO:teuthology.orchestra.run.trial150.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 93/96 2026-02-21T23:06:35.051 INFO:teuthology.orchestra.run.trial150.stdout: Verifying : python3-xmlsec-1.3.13-1.el9.x86_64 94/96 2026-02-21T23:06:35.051 INFO:teuthology.orchestra.run.trial150.stdout: Verifying : python3-xmltodict-0.12.0-15.el9.noarch 95/96 2026-02-21T23:06:35.106 INFO:teuthology.orchestra.run.trial150.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 96/96 2026-02-21T23:06:35.106 INFO:teuthology.orchestra.run.trial150.stdout: 2026-02-21T23:06:35.106 INFO:teuthology.orchestra.run.trial150.stdout:Installed: 2026-02-21T23:06:35.107 INFO:teuthology.orchestra.run.trial150.stdout: abseil-cpp-20211102.0-4.el9.x86_64 2026-02-21T23:06:35.107 INFO:teuthology.orchestra.run.trial150.stdout: ceph-2:20.3.0-5425.g7c8cf961.el9.x86_64 2026-02-21T23:06:35.107 INFO:teuthology.orchestra.run.trial150.stdout: ceph-grafana-dashboards-2:20.3.0-5425.g7c8cf961.el9.noarch 2026-02-21T23:06:35.107 INFO:teuthology.orchestra.run.trial150.stdout: ceph-mds-2:20.3.0-5425.g7c8cf961.el9.x86_64 2026-02-21T23:06:35.107 INFO:teuthology.orchestra.run.trial150.stdout: ceph-mgr-2:20.3.0-5425.g7c8cf961.el9.x86_64 2026-02-21T23:06:35.107 INFO:teuthology.orchestra.run.trial150.stdout: ceph-mgr-cephadm-2:20.3.0-5425.g7c8cf961.el9.noarch 2026-02-21T23:06:35.107 INFO:teuthology.orchestra.run.trial150.stdout: ceph-mgr-dashboard-2:20.3.0-5425.g7c8cf961.el9.noarch 2026-02-21T23:06:35.107 INFO:teuthology.orchestra.run.trial150.stdout: ceph-mgr-diskprediction-local-2:20.3.0-5425.g7c8cf961.el9.noarch 2026-02-21T23:06:35.107 INFO:teuthology.orchestra.run.trial150.stdout: ceph-mgr-k8sevents-2:20.3.0-5425.g7c8cf961.el9.noarch 2026-02-21T23:06:35.107 INFO:teuthology.orchestra.run.trial150.stdout: ceph-mgr-modules-core-2:20.3.0-5425.g7c8cf961.el9.noarch 2026-02-21T23:06:35.107 INFO:teuthology.orchestra.run.trial150.stdout: ceph-mgr-rook-2:20.3.0-5425.g7c8cf961.el9.noarch 2026-02-21T23:06:35.107 INFO:teuthology.orchestra.run.trial150.stdout: ceph-mon-2:20.3.0-5425.g7c8cf961.el9.x86_64 2026-02-21T23:06:35.107 INFO:teuthology.orchestra.run.trial150.stdout: ceph-osd-2:20.3.0-5425.g7c8cf961.el9.x86_64 2026-02-21T23:06:35.108 INFO:teuthology.orchestra.run.trial150.stdout: ceph-osd-classic-2:20.3.0-5425.g7c8cf961.el9.x86_64 2026-02-21T23:06:35.108 INFO:teuthology.orchestra.run.trial150.stdout: ceph-osd-crimson-2:20.3.0-5425.g7c8cf961.el9.x86_64 2026-02-21T23:06:35.108 INFO:teuthology.orchestra.run.trial150.stdout: ceph-prometheus-alerts-2:20.3.0-5425.g7c8cf961.el9.noarch 2026-02-21T23:06:35.108 INFO:teuthology.orchestra.run.trial150.stdout: ceph-volume-2:20.3.0-5425.g7c8cf961.el9.noarch 2026-02-21T23:06:35.108 INFO:teuthology.orchestra.run.trial150.stdout: cephadm-2:20.3.0-5425.g7c8cf961.el9.noarch 2026-02-21T23:06:35.108 INFO:teuthology.orchestra.run.trial150.stdout: cryptsetup-2.8.1-3.el9.x86_64 2026-02-21T23:06:35.108 INFO:teuthology.orchestra.run.trial150.stdout: flexiblas-3.0.4-9.el9.x86_64 2026-02-21T23:06:35.108 INFO:teuthology.orchestra.run.trial150.stdout: flexiblas-netlib-3.0.4-9.el9.x86_64 2026-02-21T23:06:35.108 INFO:teuthology.orchestra.run.trial150.stdout: flexiblas-openblas-openmp-3.0.4-9.el9.x86_64 2026-02-21T23:06:35.108 INFO:teuthology.orchestra.run.trial150.stdout: grpc-data-1.46.7-10.el9.noarch 2026-02-21T23:06:35.108 INFO:teuthology.orchestra.run.trial150.stdout: ledmon-libs-1.1.0-3.el9.x86_64 2026-02-21T23:06:35.108 INFO:teuthology.orchestra.run.trial150.stdout: libconfig-1.7.2-9.el9.x86_64 2026-02-21T23:06:35.108 INFO:teuthology.orchestra.run.trial150.stdout: libgfortran-11.5.0-14.el9.x86_64 2026-02-21T23:06:35.109 INFO:teuthology.orchestra.run.trial150.stdout: libquadmath-11.5.0-14.el9.x86_64 2026-02-21T23:06:35.109 INFO:teuthology.orchestra.run.trial150.stdout: libstoragemgmt-1.10.1-1.el9.x86_64 2026-02-21T23:06:35.109 INFO:teuthology.orchestra.run.trial150.stdout: lua-5.4.4-4.el9.x86_64 2026-02-21T23:06:35.109 INFO:teuthology.orchestra.run.trial150.stdout: lua-devel-5.4.4-4.el9.x86_64 2026-02-21T23:06:35.109 INFO:teuthology.orchestra.run.trial150.stdout: luarocks-3.9.2-5.el9.noarch 2026-02-21T23:06:35.109 INFO:teuthology.orchestra.run.trial150.stdout: openblas-0.3.29-1.el9.x86_64 2026-02-21T23:06:35.109 INFO:teuthology.orchestra.run.trial150.stdout: openblas-openmp-0.3.29-1.el9.x86_64 2026-02-21T23:06:35.109 INFO:teuthology.orchestra.run.trial150.stdout: protobuf-compiler-3.14.0-17.el9.x86_64 2026-02-21T23:06:35.109 INFO:teuthology.orchestra.run.trial150.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2026-02-21T23:06:35.109 INFO:teuthology.orchestra.run.trial150.stdout: python3-autocommand-2.2.2-8.el9.noarch 2026-02-21T23:06:35.109 INFO:teuthology.orchestra.run.trial150.stdout: python3-babel-2.9.1-2.el9.noarch 2026-02-21T23:06:35.109 INFO:teuthology.orchestra.run.trial150.stdout: python3-backports-tarfile-1.2.0-1.el9.noarch 2026-02-21T23:06:35.109 INFO:teuthology.orchestra.run.trial150.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2026-02-21T23:06:35.109 INFO:teuthology.orchestra.run.trial150.stdout: python3-cachetools-4.2.4-1.el9.noarch 2026-02-21T23:06:35.109 INFO:teuthology.orchestra.run.trial150.stdout: python3-certifi-2023.05.07-4.el9.noarch 2026-02-21T23:06:35.109 INFO:teuthology.orchestra.run.trial150.stdout: python3-cffi-1.14.5-5.el9.x86_64 2026-02-21T23:06:35.109 INFO:teuthology.orchestra.run.trial150.stdout: python3-cheroot-10.0.1-4.el9.noarch 2026-02-21T23:06:35.110 INFO:teuthology.orchestra.run.trial150.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2026-02-21T23:06:35.110 INFO:teuthology.orchestra.run.trial150.stdout: python3-cryptography-36.0.1-5.el9.x86_64 2026-02-21T23:06:35.110 INFO:teuthology.orchestra.run.trial150.stdout: python3-devel-3.9.25-3.el9.x86_64 2026-02-21T23:06:35.110 INFO:teuthology.orchestra.run.trial150.stdout: python3-google-auth-1:2.45.0-1.el9.noarch 2026-02-21T23:06:35.110 INFO:teuthology.orchestra.run.trial150.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2026-02-21T23:06:35.110 INFO:teuthology.orchestra.run.trial150.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2026-02-21T23:06:35.110 INFO:teuthology.orchestra.run.trial150.stdout: python3-influxdb-5.3.1-1.el9.noarch 2026-02-21T23:06:35.110 INFO:teuthology.orchestra.run.trial150.stdout: python3-isodate-0.6.1-3.el9.noarch 2026-02-21T23:06:35.110 INFO:teuthology.orchestra.run.trial150.stdout: python3-jaraco-8.2.1-3.el9.noarch 2026-02-21T23:06:35.110 INFO:teuthology.orchestra.run.trial150.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2026-02-21T23:06:35.110 INFO:teuthology.orchestra.run.trial150.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2026-02-21T23:06:35.110 INFO:teuthology.orchestra.run.trial150.stdout: python3-jaraco-context-6.0.1-3.el9.noarch 2026-02-21T23:06:35.110 INFO:teuthology.orchestra.run.trial150.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2026-02-21T23:06:35.110 INFO:teuthology.orchestra.run.trial150.stdout: python3-jaraco-text-4.0.0-2.el9.noarch 2026-02-21T23:06:35.110 INFO:teuthology.orchestra.run.trial150.stdout: python3-jinja2-2.11.3-8.el9.noarch 2026-02-21T23:06:35.110 INFO:teuthology.orchestra.run.trial150.stdout: python3-jmespath-1.0.1-1.el9.noarch 2026-02-21T23:06:35.111 INFO:teuthology.orchestra.run.trial150.stdout: python3-kubernetes-1:26.1.0-3.el9.noarch 2026-02-21T23:06:35.111 INFO:teuthology.orchestra.run.trial150.stdout: python3-libstoragemgmt-1.10.1-1.el9.x86_64 2026-02-21T23:06:35.111 INFO:teuthology.orchestra.run.trial150.stdout: python3-lxml-4.6.5-3.el9.x86_64 2026-02-21T23:06:35.111 INFO:teuthology.orchestra.run.trial150.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2026-02-21T23:06:35.111 INFO:teuthology.orchestra.run.trial150.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2026-02-21T23:06:35.111 INFO:teuthology.orchestra.run.trial150.stdout: python3-msgpack-1.0.3-2.el9.x86_64 2026-02-21T23:06:35.111 INFO:teuthology.orchestra.run.trial150.stdout: python3-natsort-7.1.1-5.el9.noarch 2026-02-21T23:06:35.111 INFO:teuthology.orchestra.run.trial150.stdout: python3-numpy-1:1.23.5-2.el9.x86_64 2026-02-21T23:06:35.111 INFO:teuthology.orchestra.run.trial150.stdout: python3-numpy-f2py-1:1.23.5-2.el9.x86_64 2026-02-21T23:06:35.111 INFO:teuthology.orchestra.run.trial150.stdout: python3-packaging-20.9-5.el9.noarch 2026-02-21T23:06:35.111 INFO:teuthology.orchestra.run.trial150.stdout: python3-ply-3.11-14.el9.noarch 2026-02-21T23:06:35.111 INFO:teuthology.orchestra.run.trial150.stdout: python3-portend-3.1.0-2.el9.noarch 2026-02-21T23:06:35.111 INFO:teuthology.orchestra.run.trial150.stdout: python3-protobuf-3.14.0-17.el9.noarch 2026-02-21T23:06:35.111 INFO:teuthology.orchestra.run.trial150.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2026-02-21T23:06:35.111 INFO:teuthology.orchestra.run.trial150.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2026-02-21T23:06:35.111 INFO:teuthology.orchestra.run.trial150.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2026-02-21T23:06:35.111 INFO:teuthology.orchestra.run.trial150.stdout: python3-pycparser-2.20-6.el9.noarch 2026-02-21T23:06:35.112 INFO:teuthology.orchestra.run.trial150.stdout: python3-pyparsing-2.4.7-9.el9.noarch 2026-02-21T23:06:35.112 INFO:teuthology.orchestra.run.trial150.stdout: python3-repoze-lru-0.7-16.el9.noarch 2026-02-21T23:06:35.112 INFO:teuthology.orchestra.run.trial150.stdout: python3-requests-2.25.1-10.el9.noarch 2026-02-21T23:06:35.112 INFO:teuthology.orchestra.run.trial150.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2026-02-21T23:06:35.112 INFO:teuthology.orchestra.run.trial150.stdout: python3-routes-2.5.1-5.el9.noarch 2026-02-21T23:06:35.112 INFO:teuthology.orchestra.run.trial150.stdout: python3-rsa-4.9-2.el9.noarch 2026-02-21T23:06:35.112 INFO:teuthology.orchestra.run.trial150.stdout: python3-saml-1.16.0-1.el9.noarch 2026-02-21T23:06:35.112 INFO:teuthology.orchestra.run.trial150.stdout: python3-scipy-1.9.3-2.el9.x86_64 2026-02-21T23:06:35.112 INFO:teuthology.orchestra.run.trial150.stdout: python3-tempora-5.0.0-2.el9.noarch 2026-02-21T23:06:35.112 INFO:teuthology.orchestra.run.trial150.stdout: python3-toml-0.10.2-6.el9.noarch 2026-02-21T23:06:35.112 INFO:teuthology.orchestra.run.trial150.stdout: python3-typing-extensions-4.15.0-1.el9.noarch 2026-02-21T23:06:35.112 INFO:teuthology.orchestra.run.trial150.stdout: python3-urllib3-1.26.5-7.el9.noarch 2026-02-21T23:06:35.112 INFO:teuthology.orchestra.run.trial150.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2026-02-21T23:06:35.112 INFO:teuthology.orchestra.run.trial150.stdout: python3-xmlsec-1.3.13-1.el9.x86_64 2026-02-21T23:06:35.112 INFO:teuthology.orchestra.run.trial150.stdout: python3-xmltodict-0.12.0-15.el9.noarch 2026-02-21T23:06:35.112 INFO:teuthology.orchestra.run.trial150.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2026-02-21T23:06:35.112 INFO:teuthology.orchestra.run.trial150.stdout: unzip-6.0-59.el9.x86_64 2026-02-21T23:06:35.112 INFO:teuthology.orchestra.run.trial150.stdout: xmlsec1-1.2.29-13.el9.x86_64 2026-02-21T23:06:35.112 INFO:teuthology.orchestra.run.trial150.stdout: xmlsec1-openssl-1.2.29-13.el9.x86_64 2026-02-21T23:06:35.113 INFO:teuthology.orchestra.run.trial150.stdout: zip-3.0-35.el9.x86_64 2026-02-21T23:06:35.113 INFO:teuthology.orchestra.run.trial150.stdout: 2026-02-21T23:06:35.113 INFO:teuthology.orchestra.run.trial150.stdout:Complete! 2026-02-21T23:06:35.170 DEBUG:teuthology.orchestra.run.trial150:> sudo yum -y install ceph-base 2026-02-21T23:06:35.486 INFO:teuthology.orchestra.run.trial150.stdout:Last metadata expiration check: 0:01:09 ago on Sat 21 Feb 2026 11:05:26 PM UTC. 2026-02-21T23:06:35.551 INFO:teuthology.orchestra.run.trial150.stdout:Package ceph-base-2:20.3.0-5425.g7c8cf961.el9.x86_64 is already installed. 2026-02-21T23:06:35.569 INFO:teuthology.orchestra.run.trial150.stdout:Dependencies resolved. 2026-02-21T23:06:35.570 INFO:teuthology.orchestra.run.trial150.stdout:Nothing to do. 2026-02-21T23:06:35.570 INFO:teuthology.orchestra.run.trial150.stdout:Complete! 2026-02-21T23:06:35.592 DEBUG:teuthology.orchestra.run.trial150:> sudo yum -y install cephadm 2026-02-21T23:06:35.902 INFO:teuthology.orchestra.run.trial150.stdout:Last metadata expiration check: 0:01:09 ago on Sat 21 Feb 2026 11:05:26 PM UTC. 2026-02-21T23:06:35.966 INFO:teuthology.orchestra.run.trial150.stdout:Package cephadm-2:20.3.0-5425.g7c8cf961.el9.noarch is already installed. 2026-02-21T23:06:35.984 INFO:teuthology.orchestra.run.trial150.stdout:Dependencies resolved. 2026-02-21T23:06:35.985 INFO:teuthology.orchestra.run.trial150.stdout:Nothing to do. 2026-02-21T23:06:35.985 INFO:teuthology.orchestra.run.trial150.stdout:Complete! 2026-02-21T23:06:36.006 DEBUG:teuthology.orchestra.run.trial150:> sudo yum -y install ceph-immutable-object-cache 2026-02-21T23:06:36.314 INFO:teuthology.orchestra.run.trial150.stdout:Last metadata expiration check: 0:01:10 ago on Sat 21 Feb 2026 11:05:26 PM UTC. 2026-02-21T23:06:36.396 INFO:teuthology.orchestra.run.trial150.stdout:Dependencies resolved. 2026-02-21T23:06:36.396 INFO:teuthology.orchestra.run.trial150.stdout:================================================================================ 2026-02-21T23:06:36.397 INFO:teuthology.orchestra.run.trial150.stdout: Package Arch Version Repo Size 2026-02-21T23:06:36.397 INFO:teuthology.orchestra.run.trial150.stdout:================================================================================ 2026-02-21T23:06:36.397 INFO:teuthology.orchestra.run.trial150.stdout:Installing: 2026-02-21T23:06:36.397 INFO:teuthology.orchestra.run.trial150.stdout: ceph-immutable-object-cache x86_64 2:20.3.0-5425.g7c8cf961.el9 ceph 147 k 2026-02-21T23:06:36.397 INFO:teuthology.orchestra.run.trial150.stdout: 2026-02-21T23:06:36.397 INFO:teuthology.orchestra.run.trial150.stdout:Transaction Summary 2026-02-21T23:06:36.397 INFO:teuthology.orchestra.run.trial150.stdout:================================================================================ 2026-02-21T23:06:36.397 INFO:teuthology.orchestra.run.trial150.stdout:Install 1 Package 2026-02-21T23:06:36.397 INFO:teuthology.orchestra.run.trial150.stdout: 2026-02-21T23:06:36.397 INFO:teuthology.orchestra.run.trial150.stdout:Total download size: 147 k 2026-02-21T23:06:36.398 INFO:teuthology.orchestra.run.trial150.stdout:Installed size: 419 k 2026-02-21T23:06:36.398 INFO:teuthology.orchestra.run.trial150.stdout:Downloading Packages: 2026-02-21T23:06:36.577 INFO:teuthology.orchestra.run.trial150.stdout:ceph-immutable-object-cache-20.3.0-5425.g7c8cf9 818 kB/s | 147 kB 00:00 2026-02-21T23:06:36.578 INFO:teuthology.orchestra.run.trial150.stdout:-------------------------------------------------------------------------------- 2026-02-21T23:06:36.578 INFO:teuthology.orchestra.run.trial150.stdout:Total 814 kB/s | 147 kB 00:00 2026-02-21T23:06:36.578 INFO:teuthology.orchestra.run.trial150.stdout:Running transaction check 2026-02-21T23:06:36.583 INFO:teuthology.orchestra.run.trial150.stdout:Transaction check succeeded. 2026-02-21T23:06:36.583 INFO:teuthology.orchestra.run.trial150.stdout:Running transaction test 2026-02-21T23:06:36.614 INFO:teuthology.orchestra.run.trial150.stdout:Transaction test succeeded. 2026-02-21T23:06:36.614 INFO:teuthology.orchestra.run.trial150.stdout:Running transaction 2026-02-21T23:06:36.669 INFO:teuthology.orchestra.run.trial150.stdout: Preparing : 1/1 2026-02-21T23:06:36.671 INFO:teuthology.orchestra.run.trial150.stdout: Installing : ceph-immutable-object-cache-2:20.3.0-5425.g7c8cf961. 1/1 2026-02-21T23:06:36.693 INFO:teuthology.orchestra.run.trial150.stdout: Running scriptlet: ceph-immutable-object-cache-2:20.3.0-5425.g7c8cf961. 1/1 2026-02-21T23:06:36.693 INFO:teuthology.orchestra.run.trial150.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-21T23:06:36.693 INFO:teuthology.orchestra.run.trial150.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2026-02-21T23:06:36.693 INFO:teuthology.orchestra.run.trial150.stdout: 2026-02-21T23:06:37.001 INFO:teuthology.orchestra.run.trial150.stdout: Verifying : ceph-immutable-object-cache-2:20.3.0-5425.g7c8cf961. 1/1 2026-02-21T23:06:37.001 INFO:teuthology.orchestra.run.trial150.stdout: 2026-02-21T23:06:37.001 INFO:teuthology.orchestra.run.trial150.stdout:Installed: 2026-02-21T23:06:37.001 INFO:teuthology.orchestra.run.trial150.stdout: ceph-immutable-object-cache-2:20.3.0-5425.g7c8cf961.el9.x86_64 2026-02-21T23:06:37.001 INFO:teuthology.orchestra.run.trial150.stdout: 2026-02-21T23:06:37.001 INFO:teuthology.orchestra.run.trial150.stdout:Complete! 2026-02-21T23:06:37.028 DEBUG:teuthology.orchestra.run.trial150:> sudo yum -y install ceph-mgr 2026-02-21T23:06:37.342 INFO:teuthology.orchestra.run.trial150.stdout:Last metadata expiration check: 0:01:11 ago on Sat 21 Feb 2026 11:05:26 PM UTC. 2026-02-21T23:06:37.406 INFO:teuthology.orchestra.run.trial150.stdout:Package ceph-mgr-2:20.3.0-5425.g7c8cf961.el9.x86_64 is already installed. 2026-02-21T23:06:37.424 INFO:teuthology.orchestra.run.trial150.stdout:Dependencies resolved. 2026-02-21T23:06:37.425 INFO:teuthology.orchestra.run.trial150.stdout:Nothing to do. 2026-02-21T23:06:37.425 INFO:teuthology.orchestra.run.trial150.stdout:Complete! 2026-02-21T23:06:37.447 DEBUG:teuthology.orchestra.run.trial150:> sudo yum -y install ceph-mgr-dashboard 2026-02-21T23:06:37.769 INFO:teuthology.orchestra.run.trial150.stdout:Last metadata expiration check: 0:01:11 ago on Sat 21 Feb 2026 11:05:26 PM UTC. 2026-02-21T23:06:37.835 INFO:teuthology.orchestra.run.trial150.stdout:Package ceph-mgr-dashboard-2:20.3.0-5425.g7c8cf961.el9.noarch is already installed. 2026-02-21T23:06:37.853 INFO:teuthology.orchestra.run.trial150.stdout:Dependencies resolved. 2026-02-21T23:06:37.853 INFO:teuthology.orchestra.run.trial150.stdout:Nothing to do. 2026-02-21T23:06:37.853 INFO:teuthology.orchestra.run.trial150.stdout:Complete! 2026-02-21T23:06:37.874 DEBUG:teuthology.orchestra.run.trial150:> sudo yum -y install ceph-mgr-diskprediction-local 2026-02-21T23:06:38.187 INFO:teuthology.orchestra.run.trial150.stdout:Last metadata expiration check: 0:01:12 ago on Sat 21 Feb 2026 11:05:26 PM UTC. 2026-02-21T23:06:38.252 INFO:teuthology.orchestra.run.trial150.stdout:Package ceph-mgr-diskprediction-local-2:20.3.0-5425.g7c8cf961.el9.noarch is already installed. 2026-02-21T23:06:38.270 INFO:teuthology.orchestra.run.trial150.stdout:Dependencies resolved. 2026-02-21T23:06:38.270 INFO:teuthology.orchestra.run.trial150.stdout:Nothing to do. 2026-02-21T23:06:38.271 INFO:teuthology.orchestra.run.trial150.stdout:Complete! 2026-02-21T23:06:38.291 DEBUG:teuthology.orchestra.run.trial150:> sudo yum -y install ceph-mgr-rook 2026-02-21T23:06:38.605 INFO:teuthology.orchestra.run.trial150.stdout:Last metadata expiration check: 0:01:12 ago on Sat 21 Feb 2026 11:05:26 PM UTC. 2026-02-21T23:06:38.670 INFO:teuthology.orchestra.run.trial150.stdout:Package ceph-mgr-rook-2:20.3.0-5425.g7c8cf961.el9.noarch is already installed. 2026-02-21T23:06:38.688 INFO:teuthology.orchestra.run.trial150.stdout:Dependencies resolved. 2026-02-21T23:06:38.689 INFO:teuthology.orchestra.run.trial150.stdout:Nothing to do. 2026-02-21T23:06:38.689 INFO:teuthology.orchestra.run.trial150.stdout:Complete! 2026-02-21T23:06:38.711 DEBUG:teuthology.orchestra.run.trial150:> sudo yum -y install ceph-mgr-cephadm 2026-02-21T23:06:39.025 INFO:teuthology.orchestra.run.trial150.stdout:Last metadata expiration check: 0:01:13 ago on Sat 21 Feb 2026 11:05:26 PM UTC. 2026-02-21T23:06:39.090 INFO:teuthology.orchestra.run.trial150.stdout:Package ceph-mgr-cephadm-2:20.3.0-5425.g7c8cf961.el9.noarch is already installed. 2026-02-21T23:06:39.108 INFO:teuthology.orchestra.run.trial150.stdout:Dependencies resolved. 2026-02-21T23:06:39.109 INFO:teuthology.orchestra.run.trial150.stdout:Nothing to do. 2026-02-21T23:06:39.109 INFO:teuthology.orchestra.run.trial150.stdout:Complete! 2026-02-21T23:06:39.131 DEBUG:teuthology.orchestra.run.trial150:> sudo yum -y install ceph-osd 2026-02-21T23:06:39.443 INFO:teuthology.orchestra.run.trial150.stdout:Last metadata expiration check: 0:01:13 ago on Sat 21 Feb 2026 11:05:26 PM UTC. 2026-02-21T23:06:39.507 INFO:teuthology.orchestra.run.trial150.stdout:Package ceph-osd-2:20.3.0-5425.g7c8cf961.el9.x86_64 is already installed. 2026-02-21T23:06:39.525 INFO:teuthology.orchestra.run.trial150.stdout:Dependencies resolved. 2026-02-21T23:06:39.526 INFO:teuthology.orchestra.run.trial150.stdout:Nothing to do. 2026-02-21T23:06:39.526 INFO:teuthology.orchestra.run.trial150.stdout:Complete! 2026-02-21T23:06:39.547 DEBUG:teuthology.orchestra.run.trial150:> sudo yum -y install ceph-osd-classic 2026-02-21T23:06:39.859 INFO:teuthology.orchestra.run.trial150.stdout:Last metadata expiration check: 0:01:13 ago on Sat 21 Feb 2026 11:05:26 PM UTC. 2026-02-21T23:06:39.924 INFO:teuthology.orchestra.run.trial150.stdout:Package ceph-osd-classic-2:20.3.0-5425.g7c8cf961.el9.x86_64 is already installed. 2026-02-21T23:06:39.943 INFO:teuthology.orchestra.run.trial150.stdout:Dependencies resolved. 2026-02-21T23:06:39.943 INFO:teuthology.orchestra.run.trial150.stdout:Nothing to do. 2026-02-21T23:06:39.943 INFO:teuthology.orchestra.run.trial150.stdout:Complete! 2026-02-21T23:06:39.964 DEBUG:teuthology.orchestra.run.trial150:> sudo yum -y install ceph-fuse 2026-02-21T23:06:40.277 INFO:teuthology.orchestra.run.trial150.stdout:Last metadata expiration check: 0:01:14 ago on Sat 21 Feb 2026 11:05:26 PM UTC. 2026-02-21T23:06:40.360 INFO:teuthology.orchestra.run.trial150.stdout:Dependencies resolved. 2026-02-21T23:06:40.360 INFO:teuthology.orchestra.run.trial150.stdout:================================================================================ 2026-02-21T23:06:40.361 INFO:teuthology.orchestra.run.trial150.stdout: Package Arch Version Repo Size 2026-02-21T23:06:40.361 INFO:teuthology.orchestra.run.trial150.stdout:================================================================================ 2026-02-21T23:06:40.361 INFO:teuthology.orchestra.run.trial150.stdout:Installing: 2026-02-21T23:06:40.361 INFO:teuthology.orchestra.run.trial150.stdout: ceph-fuse x86_64 2:20.3.0-5425.g7c8cf961.el9 ceph 987 k 2026-02-21T23:06:40.361 INFO:teuthology.orchestra.run.trial150.stdout: 2026-02-21T23:06:40.361 INFO:teuthology.orchestra.run.trial150.stdout:Transaction Summary 2026-02-21T23:06:40.361 INFO:teuthology.orchestra.run.trial150.stdout:================================================================================ 2026-02-21T23:06:40.361 INFO:teuthology.orchestra.run.trial150.stdout:Install 1 Package 2026-02-21T23:06:40.361 INFO:teuthology.orchestra.run.trial150.stdout: 2026-02-21T23:06:40.361 INFO:teuthology.orchestra.run.trial150.stdout:Total download size: 987 k 2026-02-21T23:06:40.361 INFO:teuthology.orchestra.run.trial150.stdout:Installed size: 2.9 M 2026-02-21T23:06:40.362 INFO:teuthology.orchestra.run.trial150.stdout:Downloading Packages: 2026-02-21T23:06:40.483 INFO:teuthology.orchestra.run.trial150.stdout:ceph-fuse-20.3.0-5425.g7c8cf961.el9.x86_64.rpm 8.0 MB/s | 987 kB 00:00 2026-02-21T23:06:40.483 INFO:teuthology.orchestra.run.trial150.stdout:-------------------------------------------------------------------------------- 2026-02-21T23:06:40.483 INFO:teuthology.orchestra.run.trial150.stdout:Total 7.9 MB/s | 987 kB 00:00 2026-02-21T23:06:40.483 INFO:teuthology.orchestra.run.trial150.stdout:Running transaction check 2026-02-21T23:06:40.487 INFO:teuthology.orchestra.run.trial150.stdout:Transaction check succeeded. 2026-02-21T23:06:40.488 INFO:teuthology.orchestra.run.trial150.stdout:Running transaction test 2026-02-21T23:06:40.522 INFO:teuthology.orchestra.run.trial150.stdout:Transaction test succeeded. 2026-02-21T23:06:40.522 INFO:teuthology.orchestra.run.trial150.stdout:Running transaction 2026-02-21T23:06:40.614 INFO:teuthology.orchestra.run.trial150.stdout: Preparing : 1/1 2026-02-21T23:06:40.631 INFO:teuthology.orchestra.run.trial150.stdout: Installing : ceph-fuse-2:20.3.0-5425.g7c8cf961.el9.x86_64 1/1 2026-02-21T23:06:40.878 INFO:teuthology.orchestra.run.trial150.stdout: Running scriptlet: ceph-fuse-2:20.3.0-5425.g7c8cf961.el9.x86_64 1/1 2026-02-21T23:06:40.935 INFO:teuthology.orchestra.run.trial150.stdout: Verifying : ceph-fuse-2:20.3.0-5425.g7c8cf961.el9.x86_64 1/1 2026-02-21T23:06:40.935 INFO:teuthology.orchestra.run.trial150.stdout: 2026-02-21T23:06:40.935 INFO:teuthology.orchestra.run.trial150.stdout:Installed: 2026-02-21T23:06:40.935 INFO:teuthology.orchestra.run.trial150.stdout: ceph-fuse-2:20.3.0-5425.g7c8cf961.el9.x86_64 2026-02-21T23:06:40.935 INFO:teuthology.orchestra.run.trial150.stdout: 2026-02-21T23:06:40.935 INFO:teuthology.orchestra.run.trial150.stdout:Complete! 2026-02-21T23:06:40.963 DEBUG:teuthology.orchestra.run.trial150:> sudo yum -y install ceph-volume 2026-02-21T23:06:41.281 INFO:teuthology.orchestra.run.trial150.stdout:Last metadata expiration check: 0:01:15 ago on Sat 21 Feb 2026 11:05:26 PM UTC. 2026-02-21T23:06:41.348 INFO:teuthology.orchestra.run.trial150.stdout:Package ceph-volume-2:20.3.0-5425.g7c8cf961.el9.noarch is already installed. 2026-02-21T23:06:41.366 INFO:teuthology.orchestra.run.trial150.stdout:Dependencies resolved. 2026-02-21T23:06:41.367 INFO:teuthology.orchestra.run.trial150.stdout:Nothing to do. 2026-02-21T23:06:41.367 INFO:teuthology.orchestra.run.trial150.stdout:Complete! 2026-02-21T23:06:41.388 DEBUG:teuthology.orchestra.run.trial150:> sudo yum -y install librados-devel 2026-02-21T23:06:41.700 INFO:teuthology.orchestra.run.trial150.stdout:Last metadata expiration check: 0:01:15 ago on Sat 21 Feb 2026 11:05:26 PM UTC. 2026-02-21T23:06:41.784 INFO:teuthology.orchestra.run.trial150.stdout:Dependencies resolved. 2026-02-21T23:06:41.784 INFO:teuthology.orchestra.run.trial150.stdout:================================================================================ 2026-02-21T23:06:41.784 INFO:teuthology.orchestra.run.trial150.stdout: Package Arch Version Repo Size 2026-02-21T23:06:41.784 INFO:teuthology.orchestra.run.trial150.stdout:================================================================================ 2026-02-21T23:06:41.784 INFO:teuthology.orchestra.run.trial150.stdout:Installing: 2026-02-21T23:06:41.784 INFO:teuthology.orchestra.run.trial150.stdout: librados-devel x86_64 2:20.3.0-5425.g7c8cf961.el9 ceph 123 k 2026-02-21T23:06:41.784 INFO:teuthology.orchestra.run.trial150.stdout: 2026-02-21T23:06:41.785 INFO:teuthology.orchestra.run.trial150.stdout:Transaction Summary 2026-02-21T23:06:41.785 INFO:teuthology.orchestra.run.trial150.stdout:================================================================================ 2026-02-21T23:06:41.785 INFO:teuthology.orchestra.run.trial150.stdout:Install 1 Package 2026-02-21T23:06:41.785 INFO:teuthology.orchestra.run.trial150.stdout: 2026-02-21T23:06:41.785 INFO:teuthology.orchestra.run.trial150.stdout:Total download size: 123 k 2026-02-21T23:06:41.785 INFO:teuthology.orchestra.run.trial150.stdout:Installed size: 437 k 2026-02-21T23:06:41.785 INFO:teuthology.orchestra.run.trial150.stdout:Downloading Packages: 2026-02-21T23:06:42.140 INFO:teuthology.orchestra.run.trial150.stdout:librados-devel-20.3.0-5425.g7c8cf961.el9.x86_64 347 kB/s | 123 kB 00:00 2026-02-21T23:06:42.140 INFO:teuthology.orchestra.run.trial150.stdout:-------------------------------------------------------------------------------- 2026-02-21T23:06:42.140 INFO:teuthology.orchestra.run.trial150.stdout:Total 346 kB/s | 123 kB 00:00 2026-02-21T23:06:42.140 INFO:teuthology.orchestra.run.trial150.stdout:Running transaction check 2026-02-21T23:06:42.143 INFO:teuthology.orchestra.run.trial150.stdout:Transaction check succeeded. 2026-02-21T23:06:42.143 INFO:teuthology.orchestra.run.trial150.stdout:Running transaction test 2026-02-21T23:06:42.175 INFO:teuthology.orchestra.run.trial150.stdout:Transaction test succeeded. 2026-02-21T23:06:42.176 INFO:teuthology.orchestra.run.trial150.stdout:Running transaction 2026-02-21T23:06:42.230 INFO:teuthology.orchestra.run.trial150.stdout: Preparing : 1/1 2026-02-21T23:06:42.247 INFO:teuthology.orchestra.run.trial150.stdout: Installing : librados-devel-2:20.3.0-5425.g7c8cf961.el9.x86_64 1/1 2026-02-21T23:06:42.328 INFO:teuthology.orchestra.run.trial150.stdout: Running scriptlet: librados-devel-2:20.3.0-5425.g7c8cf961.el9.x86_64 1/1 2026-02-21T23:06:42.385 INFO:teuthology.orchestra.run.trial150.stdout: Verifying : librados-devel-2:20.3.0-5425.g7c8cf961.el9.x86_64 1/1 2026-02-21T23:06:42.386 INFO:teuthology.orchestra.run.trial150.stdout: 2026-02-21T23:06:42.386 INFO:teuthology.orchestra.run.trial150.stdout:Installed: 2026-02-21T23:06:42.386 INFO:teuthology.orchestra.run.trial150.stdout: librados-devel-2:20.3.0-5425.g7c8cf961.el9.x86_64 2026-02-21T23:06:42.386 INFO:teuthology.orchestra.run.trial150.stdout: 2026-02-21T23:06:42.386 INFO:teuthology.orchestra.run.trial150.stdout:Complete! 2026-02-21T23:06:42.413 DEBUG:teuthology.orchestra.run.trial150:> sudo yum -y install libcephfs2 2026-02-21T23:06:42.728 INFO:teuthology.orchestra.run.trial150.stdout:Last metadata expiration check: 0:01:16 ago on Sat 21 Feb 2026 11:05:26 PM UTC. 2026-02-21T23:06:42.793 INFO:teuthology.orchestra.run.trial150.stdout:Package libcephfs2-2:20.3.0-5425.g7c8cf961.el9.x86_64 is already installed. 2026-02-21T23:06:42.812 INFO:teuthology.orchestra.run.trial150.stdout:Dependencies resolved. 2026-02-21T23:06:42.812 INFO:teuthology.orchestra.run.trial150.stdout:Nothing to do. 2026-02-21T23:06:42.812 INFO:teuthology.orchestra.run.trial150.stdout:Complete! 2026-02-21T23:06:42.834 DEBUG:teuthology.orchestra.run.trial150:> sudo yum -y install libcephfs-devel 2026-02-21T23:06:43.151 INFO:teuthology.orchestra.run.trial150.stdout:Last metadata expiration check: 0:01:17 ago on Sat 21 Feb 2026 11:05:26 PM UTC. 2026-02-21T23:06:43.235 INFO:teuthology.orchestra.run.trial150.stdout:Dependencies resolved. 2026-02-21T23:06:43.235 INFO:teuthology.orchestra.run.trial150.stdout:================================================================================ 2026-02-21T23:06:43.235 INFO:teuthology.orchestra.run.trial150.stdout: Package Arch Version Repo Size 2026-02-21T23:06:43.235 INFO:teuthology.orchestra.run.trial150.stdout:================================================================================ 2026-02-21T23:06:43.235 INFO:teuthology.orchestra.run.trial150.stdout:Installing: 2026-02-21T23:06:43.236 INFO:teuthology.orchestra.run.trial150.stdout: libcephfs-devel x86_64 2:20.3.0-5425.g7c8cf961.el9 ceph 37 k 2026-02-21T23:06:43.236 INFO:teuthology.orchestra.run.trial150.stdout: 2026-02-21T23:06:43.236 INFO:teuthology.orchestra.run.trial150.stdout:Transaction Summary 2026-02-21T23:06:43.236 INFO:teuthology.orchestra.run.trial150.stdout:================================================================================ 2026-02-21T23:06:43.236 INFO:teuthology.orchestra.run.trial150.stdout:Install 1 Package 2026-02-21T23:06:43.236 INFO:teuthology.orchestra.run.trial150.stdout: 2026-02-21T23:06:43.236 INFO:teuthology.orchestra.run.trial150.stdout:Total download size: 37 k 2026-02-21T23:06:43.236 INFO:teuthology.orchestra.run.trial150.stdout:Installed size: 171 k 2026-02-21T23:06:43.236 INFO:teuthology.orchestra.run.trial150.stdout:Downloading Packages: 2026-02-21T23:06:43.298 INFO:teuthology.orchestra.run.trial150.stdout:libcephfs-devel-20.3.0-5425.g7c8cf961.el9.x86_6 596 kB/s | 37 kB 00:00 2026-02-21T23:06:43.299 INFO:teuthology.orchestra.run.trial150.stdout:-------------------------------------------------------------------------------- 2026-02-21T23:06:43.299 INFO:teuthology.orchestra.run.trial150.stdout:Total 589 kB/s | 37 kB 00:00 2026-02-21T23:06:43.299 INFO:teuthology.orchestra.run.trial150.stdout:Running transaction check 2026-02-21T23:06:43.301 INFO:teuthology.orchestra.run.trial150.stdout:Transaction check succeeded. 2026-02-21T23:06:43.301 INFO:teuthology.orchestra.run.trial150.stdout:Running transaction test 2026-02-21T23:06:43.303 INFO:teuthology.orchestra.run.trial150.stdout:Transaction test succeeded. 2026-02-21T23:06:43.303 INFO:teuthology.orchestra.run.trial150.stdout:Running transaction 2026-02-21T23:06:43.325 INFO:teuthology.orchestra.run.trial150.stdout: Preparing : 1/1 2026-02-21T23:06:43.341 INFO:teuthology.orchestra.run.trial150.stdout: Installing : libcephfs-devel-2:20.3.0-5425.g7c8cf961.el9.x86_64 1/1 2026-02-21T23:06:43.400 INFO:teuthology.orchestra.run.trial150.stdout: Running scriptlet: libcephfs-devel-2:20.3.0-5425.g7c8cf961.el9.x86_64 1/1 2026-02-21T23:06:43.459 INFO:teuthology.orchestra.run.trial150.stdout: Verifying : libcephfs-devel-2:20.3.0-5425.g7c8cf961.el9.x86_64 1/1 2026-02-21T23:06:43.459 INFO:teuthology.orchestra.run.trial150.stdout: 2026-02-21T23:06:43.459 INFO:teuthology.orchestra.run.trial150.stdout:Installed: 2026-02-21T23:06:43.459 INFO:teuthology.orchestra.run.trial150.stdout: libcephfs-devel-2:20.3.0-5425.g7c8cf961.el9.x86_64 2026-02-21T23:06:43.459 INFO:teuthology.orchestra.run.trial150.stdout: 2026-02-21T23:06:43.459 INFO:teuthology.orchestra.run.trial150.stdout:Complete! 2026-02-21T23:06:43.487 DEBUG:teuthology.orchestra.run.trial150:> sudo yum -y install librados2 2026-02-21T23:06:43.796 INFO:teuthology.orchestra.run.trial150.stdout:Last metadata expiration check: 0:01:17 ago on Sat 21 Feb 2026 11:05:26 PM UTC. 2026-02-21T23:06:43.860 INFO:teuthology.orchestra.run.trial150.stdout:Package librados2-2:20.3.0-5425.g7c8cf961.el9.x86_64 is already installed. 2026-02-21T23:06:43.879 INFO:teuthology.orchestra.run.trial150.stdout:Dependencies resolved. 2026-02-21T23:06:43.879 INFO:teuthology.orchestra.run.trial150.stdout:Nothing to do. 2026-02-21T23:06:43.879 INFO:teuthology.orchestra.run.trial150.stdout:Complete! 2026-02-21T23:06:43.900 DEBUG:teuthology.orchestra.run.trial150:> sudo yum -y install librbd1 2026-02-21T23:06:44.208 INFO:teuthology.orchestra.run.trial150.stdout:Last metadata expiration check: 0:01:18 ago on Sat 21 Feb 2026 11:05:26 PM UTC. 2026-02-21T23:06:44.272 INFO:teuthology.orchestra.run.trial150.stdout:Package librbd1-2:20.3.0-5425.g7c8cf961.el9.x86_64 is already installed. 2026-02-21T23:06:44.290 INFO:teuthology.orchestra.run.trial150.stdout:Dependencies resolved. 2026-02-21T23:06:44.291 INFO:teuthology.orchestra.run.trial150.stdout:Nothing to do. 2026-02-21T23:06:44.291 INFO:teuthology.orchestra.run.trial150.stdout:Complete! 2026-02-21T23:06:44.312 DEBUG:teuthology.orchestra.run.trial150:> sudo yum -y install python3-rados 2026-02-21T23:06:44.624 INFO:teuthology.orchestra.run.trial150.stdout:Last metadata expiration check: 0:01:18 ago on Sat 21 Feb 2026 11:05:26 PM UTC. 2026-02-21T23:06:44.689 INFO:teuthology.orchestra.run.trial150.stdout:Package python3-rados-2:20.3.0-5425.g7c8cf961.el9.x86_64 is already installed. 2026-02-21T23:06:44.708 INFO:teuthology.orchestra.run.trial150.stdout:Dependencies resolved. 2026-02-21T23:06:44.709 INFO:teuthology.orchestra.run.trial150.stdout:Nothing to do. 2026-02-21T23:06:44.709 INFO:teuthology.orchestra.run.trial150.stdout:Complete! 2026-02-21T23:06:44.731 DEBUG:teuthology.orchestra.run.trial150:> sudo yum -y install python3-rgw 2026-02-21T23:06:45.044 INFO:teuthology.orchestra.run.trial150.stdout:Last metadata expiration check: 0:01:19 ago on Sat 21 Feb 2026 11:05:26 PM UTC. 2026-02-21T23:06:45.109 INFO:teuthology.orchestra.run.trial150.stdout:Package python3-rgw-2:20.3.0-5425.g7c8cf961.el9.x86_64 is already installed. 2026-02-21T23:06:45.127 INFO:teuthology.orchestra.run.trial150.stdout:Dependencies resolved. 2026-02-21T23:06:45.128 INFO:teuthology.orchestra.run.trial150.stdout:Nothing to do. 2026-02-21T23:06:45.128 INFO:teuthology.orchestra.run.trial150.stdout:Complete! 2026-02-21T23:06:45.149 DEBUG:teuthology.orchestra.run.trial150:> sudo yum -y install python3-cephfs 2026-02-21T23:06:45.463 INFO:teuthology.orchestra.run.trial150.stdout:Last metadata expiration check: 0:01:19 ago on Sat 21 Feb 2026 11:05:26 PM UTC. 2026-02-21T23:06:45.528 INFO:teuthology.orchestra.run.trial150.stdout:Package python3-cephfs-2:20.3.0-5425.g7c8cf961.el9.x86_64 is already installed. 2026-02-21T23:06:45.546 INFO:teuthology.orchestra.run.trial150.stdout:Dependencies resolved. 2026-02-21T23:06:45.547 INFO:teuthology.orchestra.run.trial150.stdout:Nothing to do. 2026-02-21T23:06:45.547 INFO:teuthology.orchestra.run.trial150.stdout:Complete! 2026-02-21T23:06:45.569 DEBUG:teuthology.orchestra.run.trial150:> sudo yum -y install python3-rbd 2026-02-21T23:06:45.883 INFO:teuthology.orchestra.run.trial150.stdout:Last metadata expiration check: 0:01:19 ago on Sat 21 Feb 2026 11:05:26 PM UTC. 2026-02-21T23:06:45.948 INFO:teuthology.orchestra.run.trial150.stdout:Package python3-rbd-2:20.3.0-5425.g7c8cf961.el9.x86_64 is already installed. 2026-02-21T23:06:45.967 INFO:teuthology.orchestra.run.trial150.stdout:Dependencies resolved. 2026-02-21T23:06:45.967 INFO:teuthology.orchestra.run.trial150.stdout:Nothing to do. 2026-02-21T23:06:45.967 INFO:teuthology.orchestra.run.trial150.stdout:Complete! 2026-02-21T23:06:45.989 DEBUG:teuthology.orchestra.run.trial150:> sudo yum -y install rbd-fuse 2026-02-21T23:06:46.302 INFO:teuthology.orchestra.run.trial150.stdout:Last metadata expiration check: 0:01:20 ago on Sat 21 Feb 2026 11:05:26 PM UTC. 2026-02-21T23:06:46.385 INFO:teuthology.orchestra.run.trial150.stdout:Dependencies resolved. 2026-02-21T23:06:46.385 INFO:teuthology.orchestra.run.trial150.stdout:================================================================================ 2026-02-21T23:06:46.385 INFO:teuthology.orchestra.run.trial150.stdout: Package Architecture Version Repository Size 2026-02-21T23:06:46.385 INFO:teuthology.orchestra.run.trial150.stdout:================================================================================ 2026-02-21T23:06:46.385 INFO:teuthology.orchestra.run.trial150.stdout:Installing: 2026-02-21T23:06:46.385 INFO:teuthology.orchestra.run.trial150.stdout: rbd-fuse x86_64 2:20.3.0-5425.g7c8cf961.el9 ceph 84 k 2026-02-21T23:06:46.385 INFO:teuthology.orchestra.run.trial150.stdout: 2026-02-21T23:06:46.385 INFO:teuthology.orchestra.run.trial150.stdout:Transaction Summary 2026-02-21T23:06:46.385 INFO:teuthology.orchestra.run.trial150.stdout:================================================================================ 2026-02-21T23:06:46.386 INFO:teuthology.orchestra.run.trial150.stdout:Install 1 Package 2026-02-21T23:06:46.386 INFO:teuthology.orchestra.run.trial150.stdout: 2026-02-21T23:06:46.386 INFO:teuthology.orchestra.run.trial150.stdout:Total download size: 84 k 2026-02-21T23:06:46.386 INFO:teuthology.orchestra.run.trial150.stdout:Installed size: 209 k 2026-02-21T23:06:46.386 INFO:teuthology.orchestra.run.trial150.stdout:Downloading Packages: 2026-02-21T23:06:46.461 INFO:teuthology.orchestra.run.trial150.stdout:rbd-fuse-20.3.0-5425.g7c8cf961.el9.x86_64.rpm 1.1 MB/s | 84 kB 00:00 2026-02-21T23:06:46.461 INFO:teuthology.orchestra.run.trial150.stdout:-------------------------------------------------------------------------------- 2026-02-21T23:06:46.461 INFO:teuthology.orchestra.run.trial150.stdout:Total 1.1 MB/s | 84 kB 00:00 2026-02-21T23:06:46.462 INFO:teuthology.orchestra.run.trial150.stdout:Running transaction check 2026-02-21T23:06:46.465 INFO:teuthology.orchestra.run.trial150.stdout:Transaction check succeeded. 2026-02-21T23:06:46.465 INFO:teuthology.orchestra.run.trial150.stdout:Running transaction test 2026-02-21T23:06:46.499 INFO:teuthology.orchestra.run.trial150.stdout:Transaction test succeeded. 2026-02-21T23:06:46.499 INFO:teuthology.orchestra.run.trial150.stdout:Running transaction 2026-02-21T23:06:46.550 INFO:teuthology.orchestra.run.trial150.stdout: Preparing : 1/1 2026-02-21T23:06:46.567 INFO:teuthology.orchestra.run.trial150.stdout: Installing : rbd-fuse-2:20.3.0-5425.g7c8cf961.el9.x86_64 1/1 2026-02-21T23:06:46.647 INFO:teuthology.orchestra.run.trial150.stdout: Running scriptlet: rbd-fuse-2:20.3.0-5425.g7c8cf961.el9.x86_64 1/1 2026-02-21T23:06:46.704 INFO:teuthology.orchestra.run.trial150.stdout: Verifying : rbd-fuse-2:20.3.0-5425.g7c8cf961.el9.x86_64 1/1 2026-02-21T23:06:46.705 INFO:teuthology.orchestra.run.trial150.stdout: 2026-02-21T23:06:46.705 INFO:teuthology.orchestra.run.trial150.stdout:Installed: 2026-02-21T23:06:46.705 INFO:teuthology.orchestra.run.trial150.stdout: rbd-fuse-2:20.3.0-5425.g7c8cf961.el9.x86_64 2026-02-21T23:06:46.705 INFO:teuthology.orchestra.run.trial150.stdout: 2026-02-21T23:06:46.705 INFO:teuthology.orchestra.run.trial150.stdout:Complete! 2026-02-21T23:06:46.734 DEBUG:teuthology.orchestra.run.trial150:> sudo yum -y install rbd-mirror 2026-02-21T23:06:47.042 INFO:teuthology.orchestra.run.trial150.stdout:Last metadata expiration check: 0:01:21 ago on Sat 21 Feb 2026 11:05:26 PM UTC. 2026-02-21T23:06:47.126 INFO:teuthology.orchestra.run.trial150.stdout:Dependencies resolved. 2026-02-21T23:06:47.127 INFO:teuthology.orchestra.run.trial150.stdout:================================================================================ 2026-02-21T23:06:47.127 INFO:teuthology.orchestra.run.trial150.stdout: Package Arch Version Repo Size 2026-02-21T23:06:47.127 INFO:teuthology.orchestra.run.trial150.stdout:================================================================================ 2026-02-21T23:06:47.127 INFO:teuthology.orchestra.run.trial150.stdout:Installing: 2026-02-21T23:06:47.127 INFO:teuthology.orchestra.run.trial150.stdout: rbd-mirror x86_64 2:20.3.0-5425.g7c8cf961.el9 ceph 2.9 M 2026-02-21T23:06:47.127 INFO:teuthology.orchestra.run.trial150.stdout: 2026-02-21T23:06:47.127 INFO:teuthology.orchestra.run.trial150.stdout:Transaction Summary 2026-02-21T23:06:47.127 INFO:teuthology.orchestra.run.trial150.stdout:================================================================================ 2026-02-21T23:06:47.128 INFO:teuthology.orchestra.run.trial150.stdout:Install 1 Package 2026-02-21T23:06:47.128 INFO:teuthology.orchestra.run.trial150.stdout: 2026-02-21T23:06:47.128 INFO:teuthology.orchestra.run.trial150.stdout:Total download size: 2.9 M 2026-02-21T23:06:47.128 INFO:teuthology.orchestra.run.trial150.stdout:Installed size: 11 M 2026-02-21T23:06:47.128 INFO:teuthology.orchestra.run.trial150.stdout:Downloading Packages: 2026-02-21T23:06:47.278 INFO:teuthology.orchestra.run.trial150.stdout:rbd-mirror-20.3.0-5425.g7c8cf961.el9.x86_64.rpm 19 MB/s | 2.9 MB 00:00 2026-02-21T23:06:47.279 INFO:teuthology.orchestra.run.trial150.stdout:-------------------------------------------------------------------------------- 2026-02-21T23:06:47.280 INFO:teuthology.orchestra.run.trial150.stdout:Total 19 MB/s | 2.9 MB 00:00 2026-02-21T23:06:47.280 INFO:teuthology.orchestra.run.trial150.stdout:Running transaction check 2026-02-21T23:06:47.284 INFO:teuthology.orchestra.run.trial150.stdout:Transaction check succeeded. 2026-02-21T23:06:47.284 INFO:teuthology.orchestra.run.trial150.stdout:Running transaction test 2026-02-21T23:06:47.324 INFO:teuthology.orchestra.run.trial150.stdout:Transaction test succeeded. 2026-02-21T23:06:47.325 INFO:teuthology.orchestra.run.trial150.stdout:Running transaction 2026-02-21T23:06:47.501 INFO:teuthology.orchestra.run.trial150.stdout: Preparing : 1/1 2026-02-21T23:06:47.502 INFO:teuthology.orchestra.run.trial150.stdout: Installing : rbd-mirror-2:20.3.0-5425.g7c8cf961.el9.x86_64 1/1 2026-02-21T23:06:47.526 INFO:teuthology.orchestra.run.trial150.stdout: Running scriptlet: rbd-mirror-2:20.3.0-5425.g7c8cf961.el9.x86_64 1/1 2026-02-21T23:06:47.526 INFO:teuthology.orchestra.run.trial150.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-21T23:06:47.526 INFO:teuthology.orchestra.run.trial150.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2026-02-21T23:06:47.526 INFO:teuthology.orchestra.run.trial150.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2026-02-21T23:06:47.526 INFO:teuthology.orchestra.run.trial150.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2026-02-21T23:06:47.526 INFO:teuthology.orchestra.run.trial150.stdout: 2026-02-21T23:06:47.823 INFO:teuthology.orchestra.run.trial150.stdout: Verifying : rbd-mirror-2:20.3.0-5425.g7c8cf961.el9.x86_64 1/1 2026-02-21T23:06:47.824 INFO:teuthology.orchestra.run.trial150.stdout: 2026-02-21T23:06:47.824 INFO:teuthology.orchestra.run.trial150.stdout:Installed: 2026-02-21T23:06:47.824 INFO:teuthology.orchestra.run.trial150.stdout: rbd-mirror-2:20.3.0-5425.g7c8cf961.el9.x86_64 2026-02-21T23:06:47.824 INFO:teuthology.orchestra.run.trial150.stdout: 2026-02-21T23:06:47.824 INFO:teuthology.orchestra.run.trial150.stdout:Complete! 2026-02-21T23:06:47.851 DEBUG:teuthology.orchestra.run.trial150:> sudo yum -y install rbd-nbd 2026-02-21T23:06:48.163 INFO:teuthology.orchestra.run.trial150.stdout:Last metadata expiration check: 0:01:22 ago on Sat 21 Feb 2026 11:05:26 PM UTC. 2026-02-21T23:06:48.246 INFO:teuthology.orchestra.run.trial150.stdout:Dependencies resolved. 2026-02-21T23:06:48.247 INFO:teuthology.orchestra.run.trial150.stdout:================================================================================ 2026-02-21T23:06:48.247 INFO:teuthology.orchestra.run.trial150.stdout: Package Architecture Version Repository Size 2026-02-21T23:06:48.247 INFO:teuthology.orchestra.run.trial150.stdout:================================================================================ 2026-02-21T23:06:48.247 INFO:teuthology.orchestra.run.trial150.stdout:Installing: 2026-02-21T23:06:48.247 INFO:teuthology.orchestra.run.trial150.stdout: rbd-nbd x86_64 2:20.3.0-5425.g7c8cf961.el9 ceph 172 k 2026-02-21T23:06:48.247 INFO:teuthology.orchestra.run.trial150.stdout: 2026-02-21T23:06:48.247 INFO:teuthology.orchestra.run.trial150.stdout:Transaction Summary 2026-02-21T23:06:48.247 INFO:teuthology.orchestra.run.trial150.stdout:================================================================================ 2026-02-21T23:06:48.247 INFO:teuthology.orchestra.run.trial150.stdout:Install 1 Package 2026-02-21T23:06:48.247 INFO:teuthology.orchestra.run.trial150.stdout: 2026-02-21T23:06:48.248 INFO:teuthology.orchestra.run.trial150.stdout:Total download size: 172 k 2026-02-21T23:06:48.248 INFO:teuthology.orchestra.run.trial150.stdout:Installed size: 462 k 2026-02-21T23:06:48.248 INFO:teuthology.orchestra.run.trial150.stdout:Downloading Packages: 2026-02-21T23:06:48.332 INFO:teuthology.orchestra.run.trial150.stdout:rbd-nbd-20.3.0-5425.g7c8cf961.el9.x86_64.rpm 2.0 MB/s | 172 kB 00:00 2026-02-21T23:06:48.332 INFO:teuthology.orchestra.run.trial150.stdout:-------------------------------------------------------------------------------- 2026-02-21T23:06:48.332 INFO:teuthology.orchestra.run.trial150.stdout:Total 2.0 MB/s | 172 kB 00:00 2026-02-21T23:06:48.332 INFO:teuthology.orchestra.run.trial150.stdout:Running transaction check 2026-02-21T23:06:48.336 INFO:teuthology.orchestra.run.trial150.stdout:Transaction check succeeded. 2026-02-21T23:06:48.336 INFO:teuthology.orchestra.run.trial150.stdout:Running transaction test 2026-02-21T23:06:48.369 INFO:teuthology.orchestra.run.trial150.stdout:Transaction test succeeded. 2026-02-21T23:06:48.369 INFO:teuthology.orchestra.run.trial150.stdout:Running transaction 2026-02-21T23:06:48.424 INFO:teuthology.orchestra.run.trial150.stdout: Preparing : 1/1 2026-02-21T23:06:48.441 INFO:teuthology.orchestra.run.trial150.stdout: Installing : rbd-nbd-2:20.3.0-5425.g7c8cf961.el9.x86_64 1/1 2026-02-21T23:06:48.523 INFO:teuthology.orchestra.run.trial150.stdout: Running scriptlet: rbd-nbd-2:20.3.0-5425.g7c8cf961.el9.x86_64 1/1 2026-02-21T23:06:48.581 INFO:teuthology.orchestra.run.trial150.stdout: Verifying : rbd-nbd-2:20.3.0-5425.g7c8cf961.el9.x86_64 1/1 2026-02-21T23:06:48.581 INFO:teuthology.orchestra.run.trial150.stdout: 2026-02-21T23:06:48.582 INFO:teuthology.orchestra.run.trial150.stdout:Installed: 2026-02-21T23:06:48.582 INFO:teuthology.orchestra.run.trial150.stdout: rbd-nbd-2:20.3.0-5425.g7c8cf961.el9.x86_64 2026-02-21T23:06:48.582 INFO:teuthology.orchestra.run.trial150.stdout: 2026-02-21T23:06:48.582 INFO:teuthology.orchestra.run.trial150.stdout:Complete! 2026-02-21T23:06:48.609 DEBUG:teuthology.parallel:result is None 2026-02-21T23:06:48.609 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=7c8cf961761b37084e637a305d42f77c52437802 2026-02-21T23:06:48.758 DEBUG:teuthology.orchestra.run.trial150:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2026-02-21T23:06:48.779 INFO:teuthology.orchestra.run.trial150.stdout:20.3.0-5425.g7c8cf961.el9 2026-02-21T23:06:48.780 INFO:teuthology.packaging:The installed version of ceph is 20.3.0-5425.g7c8cf961.el9 2026-02-21T23:06:48.780 INFO:teuthology.task.install:The correct ceph version 20.3.0-5425.g7c8cf961 is installed. 2026-02-21T23:06:48.783 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2026-02-21T23:06:48.783 DEBUG:teuthology.orchestra.run.trial150:> set -ex 2026-02-21T23:06:48.783 DEBUG:teuthology.orchestra.run.trial150:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2026-02-21T23:06:48.845 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2026-02-21T23:06:48.845 DEBUG:teuthology.orchestra.run.trial150:> set -ex 2026-02-21T23:06:48.845 DEBUG:teuthology.orchestra.run.trial150:> sudo dd of=/usr/bin/daemon-helper 2026-02-21T23:06:48.909 DEBUG:teuthology.orchestra.run.trial150:> sudo chmod a=rx -- /usr/bin/daemon-helper 2026-02-21T23:06:48.975 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2026-02-21T23:06:48.975 DEBUG:teuthology.orchestra.run.trial150:> set -ex 2026-02-21T23:06:48.975 DEBUG:teuthology.orchestra.run.trial150:> sudo dd of=/usr/bin/adjust-ulimits 2026-02-21T23:06:49.039 DEBUG:teuthology.orchestra.run.trial150:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2026-02-21T23:06:49.104 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2026-02-21T23:06:49.104 DEBUG:teuthology.orchestra.run.trial150:> set -ex 2026-02-21T23:06:49.104 DEBUG:teuthology.orchestra.run.trial150:> sudo dd of=/usr/bin/stdin-killer 2026-02-21T23:06:49.169 DEBUG:teuthology.orchestra.run.trial150:> sudo chmod a=rx -- /usr/bin/stdin-killer 2026-02-21T23:06:49.234 INFO:teuthology.run_tasks:Running task ceph... 2026-02-21T23:06:49.353 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2026-02-21T23:06:49.353 DEBUG:teuthology.orchestra.run.trial150:> sudo chmod 777 /var/log/ceph 2026-02-21T23:06:49.375 INFO:tasks.ceph:Disabling ceph logrotate... 2026-02-21T23:06:49.376 DEBUG:teuthology.orchestra.run.trial150:> sudo rm -f -- /etc/logrotate.d/ceph 2026-02-21T23:06:49.440 INFO:tasks.ceph:Creating extra log directories... 2026-02-21T23:06:49.440 DEBUG:teuthology.orchestra.run.trial150:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2026-02-21T23:06:49.513 INFO:tasks.ceph:Creating ceph cluster ceph... 2026-02-21T23:06:49.514 INFO:tasks.ceph:config {'conf': {'osd': {'osd max object name len': 400, 'osd max object namespace len': 64, 'bluestore zero block detection': True, 'debug ms': 1, 'debug osd': 20, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd mclock override recovery settings': True, 'osd mclock profile': 'high_recovery_ops', 'osd mclock skip benchmark': True, 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random'}, 'global': {'mon election default strategy': 3}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 30, 'debug ms': 1, 'debug paxos': 20, 'mon scrub interval': 300}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['but it is still running', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_APP_NOT_ENABLED\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2026-02-21T23:06:49.514 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/mnt/teuthology/nmordech-2026-02-21_21:55:34-rados-wip-rocky10-branch-of-the-day-2026-02-21-1771692874-tentacle-distro-default-trial/63256', 'branch': 'wip-rocky10-branch-of-the-day-2026-02-21-1771692874-tentacle', 'description': 'rados/singleton-nomsgr/{all/health-warnings mon_election/connectivity rados supported-random-distro$/{centos_latest}}', 'email': None, 'first_in_suite': False, 'flavor': 'default', 'job_id': '63256', 'kernel': {'branch': 'distro', 'kdb': 1, 'sha1': 'distro'}, 'last_in_suite': False, 'machine_type': 'trial', 'name': 'nmordech-2026-02-21_21:55:34-rados-wip-rocky10-branch-of-the-day-2026-02-21-1771692874-tentacle-distro-default-trial', 'no_nested_subset': False, 'os_type': 'centos', 'os_version': '9.stream', 'overrides': {'admin_socket': {'branch': 'wip-rocky10-branch-of-the-day-2026-02-21-1771692874-tentacle'}, 'ceph': {'conf': {'global': {'mon election default strategy': 3}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 30, 'debug ms': 1, 'debug paxos': 20, 'mon scrub interval': 300}, 'osd': {'bluestore zero block detection': True, 'debug ms': 1, 'debug osd': 20, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd mclock override recovery settings': True, 'osd mclock profile': 'high_recovery_ops', 'osd mclock skip benchmark': True, 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random'}}, 'flavor': 'default', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)'], 'sha1': '7c8cf961761b37084e637a305d42f77c52437802'}, 'ceph-deploy': {'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}}}, 'cephadm': {'distro-suffix': 'rockylinux-10'}, 'install': {'ceph': {'flavor': 'default', 'sha1': '7c8cf961761b37084e637a305d42f77c52437802'}}, 'workunit': {'branch': 'main', 'sha1': '53f8441172c46e34523029ab88f9db7a45514e84'}}, 'owner': 'scheduled_nmordech@soko04.front.sepia.ceph.com', 'priority': 10, 'repo': 'https://git.ceph.com/ceph-ci.git', 'roles': [['mon.a', 'mgr.x', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'osd.8', 'osd.9', 'client.0']], 'seed': 1998, 'sha1': '7c8cf961761b37084e637a305d42f77c52437802', 'sleep_before_teardown': 0, 'subset': '111/120000', 'suite': 'rados', 'suite_branch': 'main', 'suite_path': '/home/teuthworker/src/github.com_NitzanMordhai_ceph_53f8441172c46e34523029ab88f9db7a45514e84/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://github.com/NitzanMordhai/ceph.git', 'suite_sha1': '53f8441172c46e34523029ab88f9db7a45514e84', 'targets': {'trial150.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDfLMdMUlu8wAems/NUkpdaHQ49xQxHGIEb/oAcsKFOv0SadhbslhCgXiykZWo7B1FgOXC2TFblnUWpu2UjoQmI='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'branch': 'distro', 'kdb': 1, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': None}, {'ceph': {'conf': {'osd': {'osd max object name len': 400, 'osd max object namespace len': 64, 'bluestore zero block detection': True, 'debug ms': 1, 'debug osd': 20, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd mclock override recovery settings': True, 'osd mclock profile': 'high_recovery_ops', 'osd mclock skip benchmark': True, 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random'}, 'global': {'mon election default strategy': 3}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 30, 'debug ms': 1, 'debug paxos': 20, 'mon scrub interval': 300}}, 'log-ignorelist': ['but it is still running', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_APP_NOT_ENABLED\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)'], 'pre-mgr-commands': ['sudo ceph config set mgr mgr_pool false --force'], 'flavor': 'default', 'sha1': '7c8cf961761b37084e637a305d42f77c52437802', 'cluster': 'ceph'}}, {'workunit': {'clients': {'all': ['rados/test_health_warnings.sh']}}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': []}, 'teuthology_branch': 'main', 'teuthology_sha1': 'fa17720d0088c3ac28e473468bfc79eeaff5cd38', 'timestamp': '2026-02-21_21:55:34', 'tube': 'trial', 'user': 'nmordech', 'verbose': False, 'worker_log': '/home/teuthworker/mnt/teuthology/worker_logs/dispatcher.trial.1540699'} 2026-02-21T23:06:49.514 DEBUG:teuthology.orchestra.run.trial150:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2026-02-21T23:06:49.561 DEBUG:teuthology.orchestra.run.trial150:> sudo install -d -m0777 -- /var/run/ceph 2026-02-21T23:06:49.628 DEBUG:teuthology.orchestra.run.trial150:> set -ex 2026-02-21T23:06:49.628 DEBUG:teuthology.orchestra.run.trial150:> dd if=/scratch_devs of=/dev/stdout 2026-02-21T23:06:49.686 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2026-02-21T23:06:49.687 DEBUG:teuthology.orchestra.run.trial150:> stat /dev/vg_nvme/lv_1 2026-02-21T23:06:49.745 INFO:teuthology.orchestra.run.trial150.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2026-02-21T23:06:49.746 INFO:teuthology.orchestra.run.trial150.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2026-02-21T23:06:49.746 INFO:teuthology.orchestra.run.trial150.stdout:Device: 6h/6d Inode: 1308 Links: 1 2026-02-21T23:06:49.746 INFO:teuthology.orchestra.run.trial150.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2026-02-21T23:06:49.746 INFO:teuthology.orchestra.run.trial150.stdout:Context: system_u:object_r:device_t:s0 2026-02-21T23:06:49.746 INFO:teuthology.orchestra.run.trial150.stdout:Access: 2026-02-21 23:06:47.669588850 +0000 2026-02-21T23:06:49.746 INFO:teuthology.orchestra.run.trial150.stdout:Modify: 2026-02-21 23:05:41.512496729 +0000 2026-02-21T23:06:49.746 INFO:teuthology.orchestra.run.trial150.stdout:Change: 2026-02-21 23:05:41.512496729 +0000 2026-02-21T23:06:49.746 INFO:teuthology.orchestra.run.trial150.stdout: Birth: 2026-02-21 23:05:41.512496729 +0000 2026-02-21T23:06:49.747 DEBUG:teuthology.orchestra.run.trial150:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2026-02-21T23:06:49.808 INFO:teuthology.orchestra.run.trial150.stderr:1+0 records in 2026-02-21T23:06:49.808 INFO:teuthology.orchestra.run.trial150.stderr:1+0 records out 2026-02-21T23:06:49.808 INFO:teuthology.orchestra.run.trial150.stderr:512 bytes copied, 0.000147456 s, 3.5 MB/s 2026-02-21T23:06:49.809 DEBUG:teuthology.orchestra.run.trial150:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2026-02-21T23:06:49.867 DEBUG:teuthology.orchestra.run.trial150:> stat /dev/vg_nvme/lv_2 2026-02-21T23:06:49.923 INFO:teuthology.orchestra.run.trial150.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2026-02-21T23:06:49.923 INFO:teuthology.orchestra.run.trial150.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2026-02-21T23:06:49.923 INFO:teuthology.orchestra.run.trial150.stdout:Device: 6h/6d Inode: 1303 Links: 1 2026-02-21T23:06:49.923 INFO:teuthology.orchestra.run.trial150.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2026-02-21T23:06:49.924 INFO:teuthology.orchestra.run.trial150.stdout:Context: system_u:object_r:device_t:s0 2026-02-21T23:06:49.924 INFO:teuthology.orchestra.run.trial150.stdout:Access: 2026-02-21 23:06:47.670588841 +0000 2026-02-21T23:06:49.924 INFO:teuthology.orchestra.run.trial150.stdout:Modify: 2026-02-21 23:05:41.511496752 +0000 2026-02-21T23:06:49.924 INFO:teuthology.orchestra.run.trial150.stdout:Change: 2026-02-21 23:05:41.511496752 +0000 2026-02-21T23:06:49.924 INFO:teuthology.orchestra.run.trial150.stdout: Birth: 2026-02-21 23:05:41.511496752 +0000 2026-02-21T23:06:49.924 DEBUG:teuthology.orchestra.run.trial150:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2026-02-21T23:06:49.986 INFO:teuthology.orchestra.run.trial150.stderr:1+0 records in 2026-02-21T23:06:49.986 INFO:teuthology.orchestra.run.trial150.stderr:1+0 records out 2026-02-21T23:06:49.986 INFO:teuthology.orchestra.run.trial150.stderr:512 bytes copied, 0.000145082 s, 3.5 MB/s 2026-02-21T23:06:49.987 DEBUG:teuthology.orchestra.run.trial150:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2026-02-21T23:06:50.043 DEBUG:teuthology.orchestra.run.trial150:> stat /dev/vg_nvme/lv_3 2026-02-21T23:06:50.101 INFO:teuthology.orchestra.run.trial150.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2026-02-21T23:06:50.101 INFO:teuthology.orchestra.run.trial150.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2026-02-21T23:06:50.101 INFO:teuthology.orchestra.run.trial150.stdout:Device: 6h/6d Inode: 1292 Links: 1 2026-02-21T23:06:50.101 INFO:teuthology.orchestra.run.trial150.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2026-02-21T23:06:50.101 INFO:teuthology.orchestra.run.trial150.stdout:Context: system_u:object_r:device_t:s0 2026-02-21T23:06:50.101 INFO:teuthology.orchestra.run.trial150.stdout:Access: 2026-02-21 23:06:47.670588841 +0000 2026-02-21T23:06:50.101 INFO:teuthology.orchestra.run.trial150.stdout:Modify: 2026-02-21 23:05:41.511496752 +0000 2026-02-21T23:06:50.101 INFO:teuthology.orchestra.run.trial150.stdout:Change: 2026-02-21 23:05:41.511496752 +0000 2026-02-21T23:06:50.101 INFO:teuthology.orchestra.run.trial150.stdout: Birth: 2026-02-21 23:05:41.511496752 +0000 2026-02-21T23:06:50.101 DEBUG:teuthology.orchestra.run.trial150:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2026-02-21T23:06:50.164 INFO:teuthology.orchestra.run.trial150.stderr:1+0 records in 2026-02-21T23:06:50.164 INFO:teuthology.orchestra.run.trial150.stderr:1+0 records out 2026-02-21T23:06:50.164 INFO:teuthology.orchestra.run.trial150.stderr:512 bytes copied, 0.000157946 s, 3.2 MB/s 2026-02-21T23:06:50.165 DEBUG:teuthology.orchestra.run.trial150:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2026-02-21T23:06:50.222 DEBUG:teuthology.orchestra.run.trial150:> stat /dev/vg_nvme/lv_4 2026-02-21T23:06:50.278 INFO:teuthology.orchestra.run.trial150.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2026-02-21T23:06:50.278 INFO:teuthology.orchestra.run.trial150.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2026-02-21T23:06:50.278 INFO:teuthology.orchestra.run.trial150.stdout:Device: 6h/6d Inode: 1287 Links: 1 2026-02-21T23:06:50.278 INFO:teuthology.orchestra.run.trial150.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2026-02-21T23:06:50.278 INFO:teuthology.orchestra.run.trial150.stdout:Context: system_u:object_r:device_t:s0 2026-02-21T23:06:50.278 INFO:teuthology.orchestra.run.trial150.stdout:Access: 2026-02-21 23:06:47.670588841 +0000 2026-02-21T23:06:50.278 INFO:teuthology.orchestra.run.trial150.stdout:Modify: 2026-02-21 23:05:41.511496752 +0000 2026-02-21T23:06:50.278 INFO:teuthology.orchestra.run.trial150.stdout:Change: 2026-02-21 23:05:41.511496752 +0000 2026-02-21T23:06:50.279 INFO:teuthology.orchestra.run.trial150.stdout: Birth: 2026-02-21 23:05:41.511496752 +0000 2026-02-21T23:06:50.279 DEBUG:teuthology.orchestra.run.trial150:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2026-02-21T23:06:50.343 INFO:teuthology.orchestra.run.trial150.stderr:1+0 records in 2026-02-21T23:06:50.343 INFO:teuthology.orchestra.run.trial150.stderr:1+0 records out 2026-02-21T23:06:50.343 INFO:teuthology.orchestra.run.trial150.stderr:512 bytes copied, 0.000145453 s, 3.5 MB/s 2026-02-21T23:06:50.344 DEBUG:teuthology.orchestra.run.trial150:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2026-02-21T23:06:50.401 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2026-02-21T23:06:50.401 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@trial150.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}} 2026-02-21T23:06:50.402 INFO:tasks.ceph:Generating config... 2026-02-21T23:06:50.403 INFO:tasks.ceph:[osd] osd max object name len = 400 2026-02-21T23:06:50.403 INFO:tasks.ceph:[osd] osd max object namespace len = 64 2026-02-21T23:06:50.403 INFO:tasks.ceph:[osd] bluestore zero block detection = True 2026-02-21T23:06:50.403 INFO:tasks.ceph:[osd] debug ms = 1 2026-02-21T23:06:50.403 INFO:tasks.ceph:[osd] debug osd = 20 2026-02-21T23:06:50.403 INFO:tasks.ceph:[osd] osd debug verify cached snaps = True 2026-02-21T23:06:50.403 INFO:tasks.ceph:[osd] osd debug verify missing on start = True 2026-02-21T23:06:50.404 INFO:tasks.ceph:[osd] osd mclock override recovery settings = True 2026-02-21T23:06:50.404 INFO:tasks.ceph:[osd] osd mclock profile = high_recovery_ops 2026-02-21T23:06:50.404 INFO:tasks.ceph:[osd] osd mclock skip benchmark = True 2026-02-21T23:06:50.404 INFO:tasks.ceph:[osd] osd op queue = debug_random 2026-02-21T23:06:50.404 INFO:tasks.ceph:[osd] osd op queue cut off = debug_random 2026-02-21T23:06:50.404 INFO:tasks.ceph:[global] mon election default strategy = 3 2026-02-21T23:06:50.404 INFO:tasks.ceph:[mgr] debug mgr = 20 2026-02-21T23:06:50.404 INFO:tasks.ceph:[mgr] debug ms = 1 2026-02-21T23:06:50.404 INFO:tasks.ceph:[mon] debug mon = 30 2026-02-21T23:06:50.404 INFO:tasks.ceph:[mon] debug ms = 1 2026-02-21T23:06:50.404 INFO:tasks.ceph:[mon] debug paxos = 20 2026-02-21T23:06:50.404 INFO:tasks.ceph:[mon] mon scrub interval = 300 2026-02-21T23:06:50.404 INFO:tasks.ceph:Setting up mon.a... 2026-02-21T23:06:50.404 DEBUG:teuthology.orchestra.run.trial150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2026-02-21T23:06:50.474 INFO:teuthology.orchestra.run.trial150.stdout:creating /etc/ceph/ceph.keyring 2026-02-21T23:06:50.476 DEBUG:teuthology.orchestra.run.trial150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2026-02-21T23:06:50.552 DEBUG:teuthology.orchestra.run.trial150:> sudo chmod 0644 /etc/ceph/ceph.keyring 2026-02-21T23:06:50.619 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '10.20.193.150')] 2026-02-21T23:06:50.620 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=isa technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '10.20.193.150', 'mon election default strategy': 3}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd mclock skip benchmark': True, 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'osd max object name len': 400, 'osd max object namespace len': 64, 'bluestore zero block detection': True, 'debug ms': 1, 'debug osd': 20, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd mclock override recovery settings': True, 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random'}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 30, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon scrub interval': 300}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok'}, 'mon.a': {}} 2026-02-21T23:06:50.621 DEBUG:teuthology.orchestra.run.trial150:> set -ex 2026-02-21T23:06:50.621 DEBUG:teuthology.orchestra.run.trial150:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2026-02-21T23:06:50.683 DEBUG:teuthology.orchestra.run.trial150:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 10.20.193.150 --print /home/ubuntu/cephtest/ceph.monmap 2026-02-21T23:06:50.757 INFO:teuthology.orchestra.run.trial150.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2026-02-21T23:06:50.757 INFO:teuthology.orchestra.run.trial150.stdout:monmaptool: generated fsid f77161f3-2466-452f-b0a1-014035b869f9 2026-02-21T23:06:50.757 INFO:teuthology.orchestra.run.trial150.stdout:setting min_mon_release = tentacle 2026-02-21T23:06:50.757 INFO:teuthology.orchestra.run.trial150.stdout:epoch 0 2026-02-21T23:06:50.757 INFO:teuthology.orchestra.run.trial150.stdout:fsid f77161f3-2466-452f-b0a1-014035b869f9 2026-02-21T23:06:50.757 INFO:teuthology.orchestra.run.trial150.stdout:last_changed 2026-02-21T23:06:50.757757+0000 2026-02-21T23:06:50.757 INFO:teuthology.orchestra.run.trial150.stdout:created 2026-02-21T23:06:50.757757+0000 2026-02-21T23:06:50.757 INFO:teuthology.orchestra.run.trial150.stdout:min_mon_release 20 (tentacle) 2026-02-21T23:06:50.758 INFO:teuthology.orchestra.run.trial150.stdout:election_strategy: 3 2026-02-21T23:06:50.758 INFO:teuthology.orchestra.run.trial150.stdout:0: [v2:10.20.193.150:3300/0,v1:10.20.193.150:6789/0] mon.a 2026-02-21T23:06:50.758 INFO:teuthology.orchestra.run.trial150.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (1 monitors) 2026-02-21T23:06:50.758 DEBUG:teuthology.orchestra.run.trial150:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2026-02-21T23:06:50.815 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID f77161f3-2466-452f-b0a1-014035b869f9... 2026-02-21T23:06:50.816 DEBUG:teuthology.orchestra.run.trial150:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2026-02-21T23:06:50.895 INFO:teuthology.orchestra.run.trial150.stdout:[global] 2026-02-21T23:06:50.896 INFO:teuthology.orchestra.run.trial150.stdout: chdir = "" 2026-02-21T23:06:50.896 INFO:teuthology.orchestra.run.trial150.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2026-02-21T23:06:50.896 INFO:teuthology.orchestra.run.trial150.stdout: auth supported = cephx 2026-02-21T23:06:50.896 INFO:teuthology.orchestra.run.trial150.stdout: 2026-02-21T23:06:50.896 INFO:teuthology.orchestra.run.trial150.stdout: filestore xattr use omap = true 2026-02-21T23:06:50.896 INFO:teuthology.orchestra.run.trial150.stdout: 2026-02-21T23:06:50.896 INFO:teuthology.orchestra.run.trial150.stdout: mon clock drift allowed = 1.000 2026-02-21T23:06:50.896 INFO:teuthology.orchestra.run.trial150.stdout: 2026-02-21T23:06:50.896 INFO:teuthology.orchestra.run.trial150.stdout: osd crush chooseleaf type = 0 2026-02-21T23:06:50.896 INFO:teuthology.orchestra.run.trial150.stdout: auth debug = true 2026-02-21T23:06:50.896 INFO:teuthology.orchestra.run.trial150.stdout: 2026-02-21T23:06:50.896 INFO:teuthology.orchestra.run.trial150.stdout: ms die on old message = true 2026-02-21T23:06:50.896 INFO:teuthology.orchestra.run.trial150.stdout: ms die on bug = true 2026-02-21T23:06:50.896 INFO:teuthology.orchestra.run.trial150.stdout: 2026-02-21T23:06:50.896 INFO:teuthology.orchestra.run.trial150.stdout: mon max pg per osd = 10000 # >= luminous 2026-02-21T23:06:50.896 INFO:teuthology.orchestra.run.trial150.stdout: mon pg warn max object skew = 0 2026-02-21T23:06:50.896 INFO:teuthology.orchestra.run.trial150.stdout: 2026-02-21T23:06:50.896 INFO:teuthology.orchestra.run.trial150.stdout: # disable pg_autoscaler by default for new pools 2026-02-21T23:06:50.896 INFO:teuthology.orchestra.run.trial150.stdout: osd_pool_default_pg_autoscale_mode = off 2026-02-21T23:06:50.896 INFO:teuthology.orchestra.run.trial150.stdout: 2026-02-21T23:06:50.896 INFO:teuthology.orchestra.run.trial150.stdout: osd pool default size = 2 2026-02-21T23:06:50.896 INFO:teuthology.orchestra.run.trial150.stdout: 2026-02-21T23:06:50.896 INFO:teuthology.orchestra.run.trial150.stdout: mon osd allow primary affinity = true 2026-02-21T23:06:50.896 INFO:teuthology.orchestra.run.trial150.stdout: mon osd allow pg remap = true 2026-02-21T23:06:50.896 INFO:teuthology.orchestra.run.trial150.stdout: mon warn on legacy crush tunables = false 2026-02-21T23:06:50.896 INFO:teuthology.orchestra.run.trial150.stdout: mon warn on crush straw calc version zero = false 2026-02-21T23:06:50.896 INFO:teuthology.orchestra.run.trial150.stdout: mon warn on no sortbitwise = false 2026-02-21T23:06:50.896 INFO:teuthology.orchestra.run.trial150.stdout: mon warn on osd down out interval zero = false 2026-02-21T23:06:50.896 INFO:teuthology.orchestra.run.trial150.stdout: mon warn on too few osds = false 2026-02-21T23:06:50.896 INFO:teuthology.orchestra.run.trial150.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2026-02-21T23:06:50.896 INFO:teuthology.orchestra.run.trial150.stdout: mon_warn_on_pool_no_redundancy = false 2026-02-21T23:06:50.896 INFO:teuthology.orchestra.run.trial150.stdout: mon_allow_pool_size_one = true 2026-02-21T23:06:50.896 INFO:teuthology.orchestra.run.trial150.stdout: 2026-02-21T23:06:50.896 INFO:teuthology.orchestra.run.trial150.stdout: osd pool default erasure code profile = plugin=isa technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2026-02-21T23:06:50.896 INFO:teuthology.orchestra.run.trial150.stdout: 2026-02-21T23:06:50.896 INFO:teuthology.orchestra.run.trial150.stdout: osd default data pool replay window = 5 2026-02-21T23:06:50.896 INFO:teuthology.orchestra.run.trial150.stdout: 2026-02-21T23:06:50.896 INFO:teuthology.orchestra.run.trial150.stdout: mon allow pool delete = true 2026-02-21T23:06:50.896 INFO:teuthology.orchestra.run.trial150.stdout: 2026-02-21T23:06:50.896 INFO:teuthology.orchestra.run.trial150.stdout: mon cluster log file level = debug 2026-02-21T23:06:50.896 INFO:teuthology.orchestra.run.trial150.stdout: debug asserts on shutdown = true 2026-02-21T23:06:50.897 INFO:teuthology.orchestra.run.trial150.stdout: mon health detail to clog = false 2026-02-21T23:06:50.897 INFO:teuthology.orchestra.run.trial150.stdout: mon host = 10.20.193.150 2026-02-21T23:06:50.897 INFO:teuthology.orchestra.run.trial150.stdout: mon election default strategy = 3 2026-02-21T23:06:50.897 INFO:teuthology.orchestra.run.trial150.stdout: fsid = f77161f3-2466-452f-b0a1-014035b869f9 2026-02-21T23:06:50.897 INFO:teuthology.orchestra.run.trial150.stdout: 2026-02-21T23:06:50.897 INFO:teuthology.orchestra.run.trial150.stdout:[osd] 2026-02-21T23:06:50.897 INFO:teuthology.orchestra.run.trial150.stdout: osd journal size = 100 2026-02-21T23:06:50.897 INFO:teuthology.orchestra.run.trial150.stdout: 2026-02-21T23:06:50.897 INFO:teuthology.orchestra.run.trial150.stdout: osd scrub load threshold = 5.0 2026-02-21T23:06:50.897 INFO:teuthology.orchestra.run.trial150.stdout: osd scrub max interval = 600 2026-02-21T23:06:50.897 INFO:teuthology.orchestra.run.trial150.stdout: osd mclock profile = high_recovery_ops 2026-02-21T23:06:50.897 INFO:teuthology.orchestra.run.trial150.stdout: osd mclock skip benchmark = True 2026-02-21T23:06:50.897 INFO:teuthology.orchestra.run.trial150.stdout: 2026-02-21T23:06:50.897 INFO:teuthology.orchestra.run.trial150.stdout: osd recover clone overlap = true 2026-02-21T23:06:50.897 INFO:teuthology.orchestra.run.trial150.stdout: osd recovery max chunk = 1048576 2026-02-21T23:06:50.897 INFO:teuthology.orchestra.run.trial150.stdout: 2026-02-21T23:06:50.897 INFO:teuthology.orchestra.run.trial150.stdout: osd debug shutdown = true 2026-02-21T23:06:50.897 INFO:teuthology.orchestra.run.trial150.stdout: osd debug op order = true 2026-02-21T23:06:50.897 INFO:teuthology.orchestra.run.trial150.stdout: osd debug verify stray on activate = true 2026-02-21T23:06:50.897 INFO:teuthology.orchestra.run.trial150.stdout: osd debug trim objects = true 2026-02-21T23:06:50.897 INFO:teuthology.orchestra.run.trial150.stdout: 2026-02-21T23:06:50.897 INFO:teuthology.orchestra.run.trial150.stdout: osd open classes on start = true 2026-02-21T23:06:50.897 INFO:teuthology.orchestra.run.trial150.stdout: osd debug pg log writeout = true 2026-02-21T23:06:50.897 INFO:teuthology.orchestra.run.trial150.stdout: 2026-02-21T23:06:50.897 INFO:teuthology.orchestra.run.trial150.stdout: osd deep scrub update digest min age = 30 2026-02-21T23:06:50.897 INFO:teuthology.orchestra.run.trial150.stdout: 2026-02-21T23:06:50.897 INFO:teuthology.orchestra.run.trial150.stdout: osd map max advance = 10 2026-02-21T23:06:50.897 INFO:teuthology.orchestra.run.trial150.stdout: 2026-02-21T23:06:50.897 INFO:teuthology.orchestra.run.trial150.stdout: journal zero on create = true 2026-02-21T23:06:50.897 INFO:teuthology.orchestra.run.trial150.stdout: 2026-02-21T23:06:50.897 INFO:teuthology.orchestra.run.trial150.stdout: filestore ondisk finisher threads = 3 2026-02-21T23:06:50.897 INFO:teuthology.orchestra.run.trial150.stdout: filestore apply finisher threads = 3 2026-02-21T23:06:50.897 INFO:teuthology.orchestra.run.trial150.stdout: 2026-02-21T23:06:50.897 INFO:teuthology.orchestra.run.trial150.stdout: bdev debug aio = true 2026-02-21T23:06:50.897 INFO:teuthology.orchestra.run.trial150.stdout: osd debug misdirected ops = true 2026-02-21T23:06:50.897 INFO:teuthology.orchestra.run.trial150.stdout: osd max object name len = 400 2026-02-21T23:06:50.897 INFO:teuthology.orchestra.run.trial150.stdout: osd max object namespace len = 64 2026-02-21T23:06:50.897 INFO:teuthology.orchestra.run.trial150.stdout: bluestore zero block detection = True 2026-02-21T23:06:50.897 INFO:teuthology.orchestra.run.trial150.stdout: debug ms = 1 2026-02-21T23:06:50.897 INFO:teuthology.orchestra.run.trial150.stdout: debug osd = 20 2026-02-21T23:06:50.897 INFO:teuthology.orchestra.run.trial150.stdout: osd debug verify cached snaps = True 2026-02-21T23:06:50.897 INFO:teuthology.orchestra.run.trial150.stdout: osd debug verify missing on start = True 2026-02-21T23:06:50.897 INFO:teuthology.orchestra.run.trial150.stdout: osd mclock override recovery settings = True 2026-02-21T23:06:50.897 INFO:teuthology.orchestra.run.trial150.stdout: osd op queue = debug_random 2026-02-21T23:06:50.897 INFO:teuthology.orchestra.run.trial150.stdout: osd op queue cut off = debug_random 2026-02-21T23:06:50.897 INFO:teuthology.orchestra.run.trial150.stdout: 2026-02-21T23:06:50.898 INFO:teuthology.orchestra.run.trial150.stdout:[mgr] 2026-02-21T23:06:50.898 INFO:teuthology.orchestra.run.trial150.stdout: debug ms = 1 2026-02-21T23:06:50.898 INFO:teuthology.orchestra.run.trial150.stdout: debug mgr = 20 2026-02-21T23:06:50.898 INFO:teuthology.orchestra.run.trial150.stdout: debug mon = 20 2026-02-21T23:06:50.898 INFO:teuthology.orchestra.run.trial150.stdout: debug auth = 20 2026-02-21T23:06:50.898 INFO:teuthology.orchestra.run.trial150.stdout: mon reweight min pgs per osd = 4 2026-02-21T23:06:50.898 INFO:teuthology.orchestra.run.trial150.stdout: mon reweight min bytes per osd = 10 2026-02-21T23:06:50.898 INFO:teuthology.orchestra.run.trial150.stdout: mgr/telemetry/nag = false 2026-02-21T23:06:50.898 INFO:teuthology.orchestra.run.trial150.stdout: 2026-02-21T23:06:50.898 INFO:teuthology.orchestra.run.trial150.stdout:[mon] 2026-02-21T23:06:50.898 INFO:teuthology.orchestra.run.trial150.stdout: debug ms = 1 2026-02-21T23:06:50.898 INFO:teuthology.orchestra.run.trial150.stdout: debug mon = 30 2026-02-21T23:06:50.898 INFO:teuthology.orchestra.run.trial150.stdout: debug paxos = 20 2026-02-21T23:06:50.898 INFO:teuthology.orchestra.run.trial150.stdout: debug auth = 20 2026-02-21T23:06:50.898 INFO:teuthology.orchestra.run.trial150.stdout: mon data avail warn = 5 2026-02-21T23:06:50.898 INFO:teuthology.orchestra.run.trial150.stdout: mon mgr mkfs grace = 240 2026-02-21T23:06:50.898 INFO:teuthology.orchestra.run.trial150.stdout: mon reweight min pgs per osd = 4 2026-02-21T23:06:50.898 INFO:teuthology.orchestra.run.trial150.stdout: mon osd reporter subtree level = osd 2026-02-21T23:06:50.898 INFO:teuthology.orchestra.run.trial150.stdout: mon osd prime pg temp = true 2026-02-21T23:06:50.898 INFO:teuthology.orchestra.run.trial150.stdout: mon reweight min bytes per osd = 10 2026-02-21T23:06:50.898 INFO:teuthology.orchestra.run.trial150.stdout: 2026-02-21T23:06:50.898 INFO:teuthology.orchestra.run.trial150.stdout: # rotate auth tickets quickly to exercise renewal paths 2026-02-21T23:06:50.898 INFO:teuthology.orchestra.run.trial150.stdout: auth mon ticket ttl = 660 # 11m 2026-02-21T23:06:50.898 INFO:teuthology.orchestra.run.trial150.stdout: auth service ticket ttl = 240 # 4m 2026-02-21T23:06:50.898 INFO:teuthology.orchestra.run.trial150.stdout: 2026-02-21T23:06:50.898 INFO:teuthology.orchestra.run.trial150.stdout: # don't complain about insecure global_id in the test suite 2026-02-21T23:06:50.898 INFO:teuthology.orchestra.run.trial150.stdout: mon_warn_on_insecure_global_id_reclaim = false 2026-02-21T23:06:50.898 INFO:teuthology.orchestra.run.trial150.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2026-02-21T23:06:50.898 INFO:teuthology.orchestra.run.trial150.stdout: 2026-02-21T23:06:50.898 INFO:teuthology.orchestra.run.trial150.stdout: # 1m isn't quite enough 2026-02-21T23:06:50.898 INFO:teuthology.orchestra.run.trial150.stdout: mon_down_mkfs_grace = 2m 2026-02-21T23:06:50.898 INFO:teuthology.orchestra.run.trial150.stdout: 2026-02-21T23:06:50.898 INFO:teuthology.orchestra.run.trial150.stdout: mon_warn_on_filestore_osds = false 2026-02-21T23:06:50.898 INFO:teuthology.orchestra.run.trial150.stdout: mon scrub interval = 300 2026-02-21T23:06:50.898 INFO:teuthology.orchestra.run.trial150.stdout: 2026-02-21T23:06:50.898 INFO:teuthology.orchestra.run.trial150.stdout:[client] 2026-02-21T23:06:50.898 INFO:teuthology.orchestra.run.trial150.stdout: rgw cache enabled = true 2026-02-21T23:06:50.898 INFO:teuthology.orchestra.run.trial150.stdout: rgw enable ops log = true 2026-02-21T23:06:50.898 INFO:teuthology.orchestra.run.trial150.stdout: rgw enable usage log = true 2026-02-21T23:06:50.898 INFO:teuthology.orchestra.run.trial150.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2026-02-21T23:06:50.898 INFO:teuthology.orchestra.run.trial150.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2026-02-21T23:06:50.898 INFO:teuthology.orchestra.run.trial150.stdout:[mon.a] 2026-02-21T23:06:50.905 INFO:tasks.ceph:Creating admin key on mon.a... 2026-02-21T23:06:50.905 DEBUG:teuthology.orchestra.run.trial150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2026-02-21T23:06:50.986 INFO:tasks.ceph:Copying monmap to all nodes... 2026-02-21T23:06:50.986 DEBUG:teuthology.orchestra.run.trial150:> set -ex 2026-02-21T23:06:50.986 DEBUG:teuthology.orchestra.run.trial150:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2026-02-21T23:06:51.043 DEBUG:teuthology.orchestra.run.trial150:> set -ex 2026-02-21T23:06:51.044 DEBUG:teuthology.orchestra.run.trial150:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2026-02-21T23:06:51.102 INFO:tasks.ceph:Sending monmap to node ubuntu@trial150.front.sepia.ceph.com 2026-02-21T23:06:51.102 DEBUG:teuthology.orchestra.run.trial150:> set -ex 2026-02-21T23:06:51.102 DEBUG:teuthology.orchestra.run.trial150:> sudo dd of=/etc/ceph/ceph.keyring 2026-02-21T23:06:51.103 DEBUG:teuthology.orchestra.run.trial150:> sudo chmod 0644 /etc/ceph/ceph.keyring 2026-02-21T23:06:51.174 DEBUG:teuthology.orchestra.run.trial150:> set -ex 2026-02-21T23:06:51.175 DEBUG:teuthology.orchestra.run.trial150:> dd of=/home/ubuntu/cephtest/ceph.monmap 2026-02-21T23:06:51.232 INFO:tasks.ceph:Setting up mon nodes... 2026-02-21T23:06:51.233 INFO:tasks.ceph:Setting up mgr nodes... 2026-02-21T23:06:51.233 DEBUG:teuthology.orchestra.run.trial150:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2026-02-21T23:06:51.314 INFO:teuthology.orchestra.run.trial150.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2026-02-21T23:06:51.316 INFO:tasks.ceph:Setting up mds nodes... 2026-02-21T23:06:51.317 INFO:tasks.ceph_client:Setting up client nodes... 2026-02-21T23:06:51.317 DEBUG:teuthology.orchestra.run.trial150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2026-02-21T23:06:51.391 INFO:teuthology.orchestra.run.trial150.stdout:creating /etc/ceph/ceph.client.0.keyring 2026-02-21T23:06:51.402 INFO:tasks.ceph:Running mkfs on osd nodes... 2026-02-21T23:06:51.403 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@trial150.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}} 2026-02-21T23:06:51.403 DEBUG:teuthology.orchestra.run.trial150:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2026-02-21T23:06:51.466 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2026-02-21T23:06:51.466 INFO:tasks.ceph:role: osd.0 2026-02-21T23:06:51.466 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@trial150.front.sepia.ceph.com 2026-02-21T23:06:51.466 DEBUG:teuthology.orchestra.run.trial150:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2026-02-21T23:06:51.532 INFO:teuthology.orchestra.run.trial150.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=45875200 blks 2026-02-21T23:06:51.533 INFO:teuthology.orchestra.run.trial150.stdout: = sectsz=4096 attr=2, projid32bit=1 2026-02-21T23:06:51.533 INFO:teuthology.orchestra.run.trial150.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2026-02-21T23:06:51.533 INFO:teuthology.orchestra.run.trial150.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2026-02-21T23:06:51.533 INFO:teuthology.orchestra.run.trial150.stdout:data = bsize=4096 blocks=183500800, imaxpct=25 2026-02-21T23:06:51.533 INFO:teuthology.orchestra.run.trial150.stdout: = sunit=0 swidth=0 blks 2026-02-21T23:06:51.533 INFO:teuthology.orchestra.run.trial150.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2026-02-21T23:06:51.533 INFO:teuthology.orchestra.run.trial150.stdout:log =internal log bsize=4096 blocks=89600, version=2 2026-02-21T23:06:51.533 INFO:teuthology.orchestra.run.trial150.stdout: = sectsz=4096 sunit=1 blks, lazy-count=1 2026-02-21T23:06:51.533 INFO:teuthology.orchestra.run.trial150.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2026-02-21T23:06:52.105 INFO:teuthology.orchestra.run.trial150.stdout:Discarding blocks...Done. 2026-02-21T23:06:52.107 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@trial150.front.sepia.ceph.com -o noatime 2026-02-21T23:06:52.107 DEBUG:teuthology.orchestra.run.trial150:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2026-02-21T23:06:52.176 DEBUG:teuthology.orchestra.run.trial150:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2026-02-21T23:06:52.245 DEBUG:teuthology.orchestra.run.trial150:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2026-02-21T23:06:52.312 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2026-02-21T23:06:52.312 INFO:tasks.ceph:role: osd.1 2026-02-21T23:06:52.312 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@trial150.front.sepia.ceph.com 2026-02-21T23:06:52.312 DEBUG:teuthology.orchestra.run.trial150:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2026-02-21T23:06:52.377 INFO:teuthology.orchestra.run.trial150.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=45875200 blks 2026-02-21T23:06:52.377 INFO:teuthology.orchestra.run.trial150.stdout: = sectsz=4096 attr=2, projid32bit=1 2026-02-21T23:06:52.377 INFO:teuthology.orchestra.run.trial150.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2026-02-21T23:06:52.377 INFO:teuthology.orchestra.run.trial150.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2026-02-21T23:06:52.377 INFO:teuthology.orchestra.run.trial150.stdout:data = bsize=4096 blocks=183500800, imaxpct=25 2026-02-21T23:06:52.378 INFO:teuthology.orchestra.run.trial150.stdout: = sunit=0 swidth=0 blks 2026-02-21T23:06:52.378 INFO:teuthology.orchestra.run.trial150.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2026-02-21T23:06:52.378 INFO:teuthology.orchestra.run.trial150.stdout:log =internal log bsize=4096 blocks=89600, version=2 2026-02-21T23:06:52.378 INFO:teuthology.orchestra.run.trial150.stdout: = sectsz=4096 sunit=1 blks, lazy-count=1 2026-02-21T23:06:52.378 INFO:teuthology.orchestra.run.trial150.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2026-02-21T23:06:52.941 INFO:teuthology.orchestra.run.trial150.stdout:Discarding blocks...Done. 2026-02-21T23:06:52.943 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@trial150.front.sepia.ceph.com -o noatime 2026-02-21T23:06:52.943 DEBUG:teuthology.orchestra.run.trial150:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2026-02-21T23:06:53.013 DEBUG:teuthology.orchestra.run.trial150:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2026-02-21T23:06:53.080 DEBUG:teuthology.orchestra.run.trial150:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2026-02-21T23:06:53.144 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2026-02-21T23:06:53.144 INFO:tasks.ceph:role: osd.2 2026-02-21T23:06:53.144 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@trial150.front.sepia.ceph.com 2026-02-21T23:06:53.144 DEBUG:teuthology.orchestra.run.trial150:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2026-02-21T23:06:53.208 INFO:teuthology.orchestra.run.trial150.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=45875200 blks 2026-02-21T23:06:53.208 INFO:teuthology.orchestra.run.trial150.stdout: = sectsz=4096 attr=2, projid32bit=1 2026-02-21T23:06:53.208 INFO:teuthology.orchestra.run.trial150.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2026-02-21T23:06:53.208 INFO:teuthology.orchestra.run.trial150.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2026-02-21T23:06:53.208 INFO:teuthology.orchestra.run.trial150.stdout:data = bsize=4096 blocks=183500800, imaxpct=25 2026-02-21T23:06:53.208 INFO:teuthology.orchestra.run.trial150.stdout: = sunit=0 swidth=0 blks 2026-02-21T23:06:53.208 INFO:teuthology.orchestra.run.trial150.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2026-02-21T23:06:53.208 INFO:teuthology.orchestra.run.trial150.stdout:log =internal log bsize=4096 blocks=89600, version=2 2026-02-21T23:06:53.208 INFO:teuthology.orchestra.run.trial150.stdout: = sectsz=4096 sunit=1 blks, lazy-count=1 2026-02-21T23:06:53.208 INFO:teuthology.orchestra.run.trial150.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2026-02-21T23:06:53.765 INFO:teuthology.orchestra.run.trial150.stdout:Discarding blocks...Done. 2026-02-21T23:06:53.767 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@trial150.front.sepia.ceph.com -o noatime 2026-02-21T23:06:53.767 DEBUG:teuthology.orchestra.run.trial150:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2026-02-21T23:06:53.835 DEBUG:teuthology.orchestra.run.trial150:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2026-02-21T23:06:53.901 DEBUG:teuthology.orchestra.run.trial150:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2026-02-21T23:06:53.966 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2026-02-21T23:06:53.966 INFO:tasks.ceph:role: osd.3 2026-02-21T23:06:53.966 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@trial150.front.sepia.ceph.com 2026-02-21T23:06:53.966 DEBUG:teuthology.orchestra.run.trial150:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2026-02-21T23:06:54.033 INFO:teuthology.orchestra.run.trial150.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=45875200 blks 2026-02-21T23:06:54.033 INFO:teuthology.orchestra.run.trial150.stdout: = sectsz=4096 attr=2, projid32bit=1 2026-02-21T23:06:54.033 INFO:teuthology.orchestra.run.trial150.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2026-02-21T23:06:54.033 INFO:teuthology.orchestra.run.trial150.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2026-02-21T23:06:54.034 INFO:teuthology.orchestra.run.trial150.stdout:data = bsize=4096 blocks=183500800, imaxpct=25 2026-02-21T23:06:54.034 INFO:teuthology.orchestra.run.trial150.stdout: = sunit=0 swidth=0 blks 2026-02-21T23:06:54.034 INFO:teuthology.orchestra.run.trial150.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2026-02-21T23:06:54.034 INFO:teuthology.orchestra.run.trial150.stdout:log =internal log bsize=4096 blocks=89600, version=2 2026-02-21T23:06:54.034 INFO:teuthology.orchestra.run.trial150.stdout: = sectsz=4096 sunit=1 blks, lazy-count=1 2026-02-21T23:06:54.034 INFO:teuthology.orchestra.run.trial150.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2026-02-21T23:06:54.589 INFO:teuthology.orchestra.run.trial150.stdout:Discarding blocks...Done. 2026-02-21T23:06:54.591 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@trial150.front.sepia.ceph.com -o noatime 2026-02-21T23:06:54.591 DEBUG:teuthology.orchestra.run.trial150:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2026-02-21T23:06:54.660 DEBUG:teuthology.orchestra.run.trial150:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2026-02-21T23:06:54.727 DEBUG:teuthology.orchestra.run.trial150:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2026-02-21T23:06:54.792 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2026-02-21T23:06:54.792 INFO:tasks.ceph:role: osd.4 2026-02-21T23:06:54.792 DEBUG:teuthology.orchestra.run.trial150:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2026-02-21T23:06:54.856 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2026-02-21T23:06:54.856 INFO:tasks.ceph:role: osd.5 2026-02-21T23:06:54.856 DEBUG:teuthology.orchestra.run.trial150:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2026-02-21T23:06:54.919 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2026-02-21T23:06:54.919 INFO:tasks.ceph:role: osd.6 2026-02-21T23:06:54.919 DEBUG:teuthology.orchestra.run.trial150:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2026-02-21T23:06:54.983 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2026-02-21T23:06:54.983 INFO:tasks.ceph:role: osd.7 2026-02-21T23:06:54.983 DEBUG:teuthology.orchestra.run.trial150:> sudo mkdir -p /var/lib/ceph/osd/ceph-8 2026-02-21T23:06:55.047 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2026-02-21T23:06:55.047 INFO:tasks.ceph:role: osd.8 2026-02-21T23:06:55.047 DEBUG:teuthology.orchestra.run.trial150:> sudo mkdir -p /var/lib/ceph/osd/ceph-9 2026-02-21T23:06:55.112 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2026-02-21T23:06:55.112 INFO:tasks.ceph:role: osd.9 2026-02-21T23:06:55.112 DEBUG:teuthology.orchestra.run.trial150:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2026-02-21T23:06:55.189 INFO:teuthology.orchestra.run.trial150.stderr:2026-02-21T23:06:55.187+0000 7f2a0ae0f780 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2026-02-21T23:06:55.190 INFO:teuthology.orchestra.run.trial150.stderr:2026-02-21T23:06:55.187+0000 7f2a0ae0f780 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2026-02-21T23:06:55.190 INFO:teuthology.orchestra.run.trial150.stderr:2026-02-21T23:06:55.188+0000 7f2a0ae0f780 -1 bdev(0x2b5d400 /var/lib/ceph/osd/ceph-0/block) open stat got: (1) Operation not permitted 2026-02-21T23:06:55.190 INFO:teuthology.orchestra.run.trial150.stderr:2026-02-21T23:06:55.188+0000 7f2a0ae0f780 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2026-02-21T23:06:55.510 DEBUG:teuthology.orchestra.run.trial150:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2026-02-21T23:06:55.575 DEBUG:teuthology.orchestra.run.trial150:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2026-02-21T23:06:55.653 INFO:teuthology.orchestra.run.trial150.stderr:2026-02-21T23:06:55.651+0000 7f265f632780 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2026-02-21T23:06:55.653 INFO:teuthology.orchestra.run.trial150.stderr:2026-02-21T23:06:55.651+0000 7f265f632780 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2026-02-21T23:06:55.653 INFO:teuthology.orchestra.run.trial150.stderr:2026-02-21T23:06:55.651+0000 7f265f632780 -1 bdev(0x2cf7400 /var/lib/ceph/osd/ceph-1/block) open stat got: (1) Operation not permitted 2026-02-21T23:06:55.653 INFO:teuthology.orchestra.run.trial150.stderr:2026-02-21T23:06:55.651+0000 7f265f632780 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2026-02-21T23:06:55.980 DEBUG:teuthology.orchestra.run.trial150:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2026-02-21T23:06:56.044 DEBUG:teuthology.orchestra.run.trial150:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2026-02-21T23:06:56.122 INFO:teuthology.orchestra.run.trial150.stderr:2026-02-21T23:06:56.120+0000 7fc906249780 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2026-02-21T23:06:56.123 INFO:teuthology.orchestra.run.trial150.stderr:2026-02-21T23:06:56.121+0000 7fc906249780 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2026-02-21T23:06:56.123 INFO:teuthology.orchestra.run.trial150.stderr:2026-02-21T23:06:56.121+0000 7fc906249780 -1 bdev(0x270f400 /var/lib/ceph/osd/ceph-2/block) open stat got: (1) Operation not permitted 2026-02-21T23:06:56.123 INFO:teuthology.orchestra.run.trial150.stderr:2026-02-21T23:06:56.121+0000 7fc906249780 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2026-02-21T23:06:56.442 DEBUG:teuthology.orchestra.run.trial150:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2026-02-21T23:06:56.509 DEBUG:teuthology.orchestra.run.trial150:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2026-02-21T23:06:56.586 INFO:teuthology.orchestra.run.trial150.stderr:2026-02-21T23:06:56.583+0000 7fdc7e439780 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2026-02-21T23:06:56.586 INFO:teuthology.orchestra.run.trial150.stderr:2026-02-21T23:06:56.583+0000 7fdc7e439780 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2026-02-21T23:06:56.586 INFO:teuthology.orchestra.run.trial150.stderr:2026-02-21T23:06:56.583+0000 7fdc7e439780 -1 bdev(0x437f400 /var/lib/ceph/osd/ceph-3/block) open stat got: (1) Operation not permitted 2026-02-21T23:06:56.587 INFO:teuthology.orchestra.run.trial150.stderr:2026-02-21T23:06:56.584+0000 7fdc7e439780 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2026-02-21T23:06:56.894 DEBUG:teuthology.orchestra.run.trial150:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2026-02-21T23:06:56.960 DEBUG:teuthology.orchestra.run.trial150:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2026-02-21T23:06:57.037 INFO:teuthology.orchestra.run.trial150.stderr:2026-02-21T23:06:57.035+0000 7f5592fd8780 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2026-02-21T23:06:57.037 INFO:teuthology.orchestra.run.trial150.stderr:2026-02-21T23:06:57.035+0000 7f5592fd8780 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2026-02-21T23:06:57.037 INFO:teuthology.orchestra.run.trial150.stderr:2026-02-21T23:06:57.035+0000 7f5592fd8780 -1 bdev(0x25f1400 /var/lib/ceph/osd/ceph-4/block) open stat got: (1) Operation not permitted 2026-02-21T23:06:57.037 INFO:teuthology.orchestra.run.trial150.stderr:2026-02-21T23:06:57.036+0000 7f5592fd8780 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2026-02-21T23:06:57.352 DEBUG:teuthology.orchestra.run.trial150:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2026-02-21T23:06:57.418 DEBUG:teuthology.orchestra.run.trial150:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2026-02-21T23:06:57.494 INFO:teuthology.orchestra.run.trial150.stderr:2026-02-21T23:06:57.492+0000 7f1a2f25c780 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2026-02-21T23:06:57.494 INFO:teuthology.orchestra.run.trial150.stderr:2026-02-21T23:06:57.492+0000 7f1a2f25c780 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2026-02-21T23:06:57.494 INFO:teuthology.orchestra.run.trial150.stderr:2026-02-21T23:06:57.492+0000 7f1a2f25c780 -1 bdev(0x3a83400 /var/lib/ceph/osd/ceph-5/block) open stat got: (1) Operation not permitted 2026-02-21T23:06:57.494 INFO:teuthology.orchestra.run.trial150.stderr:2026-02-21T23:06:57.492+0000 7f1a2f25c780 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2026-02-21T23:06:57.804 DEBUG:teuthology.orchestra.run.trial150:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2026-02-21T23:06:57.869 DEBUG:teuthology.orchestra.run.trial150:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2026-02-21T23:06:57.945 INFO:teuthology.orchestra.run.trial150.stderr:2026-02-21T23:06:57.943+0000 7fc9dee12780 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2026-02-21T23:06:57.946 INFO:teuthology.orchestra.run.trial150.stderr:2026-02-21T23:06:57.943+0000 7fc9dee12780 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2026-02-21T23:06:57.946 INFO:teuthology.orchestra.run.trial150.stderr:2026-02-21T23:06:57.943+0000 7fc9dee12780 -1 bdev(0x37d9400 /var/lib/ceph/osd/ceph-6/block) open stat got: (1) Operation not permitted 2026-02-21T23:06:57.946 INFO:teuthology.orchestra.run.trial150.stderr:2026-02-21T23:06:57.944+0000 7fc9dee12780 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2026-02-21T23:06:58.270 DEBUG:teuthology.orchestra.run.trial150:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2026-02-21T23:06:58.336 DEBUG:teuthology.orchestra.run.trial150:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2026-02-21T23:06:58.411 INFO:teuthology.orchestra.run.trial150.stderr:2026-02-21T23:06:58.409+0000 7f869940b780 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2026-02-21T23:06:58.411 INFO:teuthology.orchestra.run.trial150.stderr:2026-02-21T23:06:58.409+0000 7f869940b780 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2026-02-21T23:06:58.412 INFO:teuthology.orchestra.run.trial150.stderr:2026-02-21T23:06:58.409+0000 7f869940b780 -1 bdev(0x2d15400 /var/lib/ceph/osd/ceph-7/block) open stat got: (1) Operation not permitted 2026-02-21T23:06:58.412 INFO:teuthology.orchestra.run.trial150.stderr:2026-02-21T23:06:58.409+0000 7f869940b780 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2026-02-21T23:06:58.722 DEBUG:teuthology.orchestra.run.trial150:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2026-02-21T23:06:58.788 DEBUG:teuthology.orchestra.run.trial150:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 8 --monmap /home/ubuntu/cephtest/ceph.monmap 2026-02-21T23:06:58.865 INFO:teuthology.orchestra.run.trial150.stderr:2026-02-21T23:06:58.862+0000 7f06c6821780 -1 auth: error reading file: /var/lib/ceph/osd/ceph-8/keyring: can't open /var/lib/ceph/osd/ceph-8/keyring: (2) No such file or directory 2026-02-21T23:06:58.865 INFO:teuthology.orchestra.run.trial150.stderr:2026-02-21T23:06:58.862+0000 7f06c6821780 -1 created new key in keyring /var/lib/ceph/osd/ceph-8/keyring 2026-02-21T23:06:58.865 INFO:teuthology.orchestra.run.trial150.stderr:2026-02-21T23:06:58.862+0000 7f06c6821780 -1 bdev(0x2f37400 /var/lib/ceph/osd/ceph-8/block) open stat got: (1) Operation not permitted 2026-02-21T23:06:58.865 INFO:teuthology.orchestra.run.trial150.stderr:2026-02-21T23:06:58.863+0000 7f06c6821780 -1 bluestore(/var/lib/ceph/osd/ceph-8) _read_fsid unparsable uuid 2026-02-21T23:06:59.191 DEBUG:teuthology.orchestra.run.trial150:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-8 2026-02-21T23:06:59.258 DEBUG:teuthology.orchestra.run.trial150:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 9 --monmap /home/ubuntu/cephtest/ceph.monmap 2026-02-21T23:06:59.333 INFO:teuthology.orchestra.run.trial150.stderr:2026-02-21T23:06:59.331+0000 7f0761e29780 -1 auth: error reading file: /var/lib/ceph/osd/ceph-9/keyring: can't open /var/lib/ceph/osd/ceph-9/keyring: (2) No such file or directory 2026-02-21T23:06:59.333 INFO:teuthology.orchestra.run.trial150.stderr:2026-02-21T23:06:59.331+0000 7f0761e29780 -1 created new key in keyring /var/lib/ceph/osd/ceph-9/keyring 2026-02-21T23:06:59.333 INFO:teuthology.orchestra.run.trial150.stderr:2026-02-21T23:06:59.331+0000 7f0761e29780 -1 bdev(0x309b400 /var/lib/ceph/osd/ceph-9/block) open stat got: (1) Operation not permitted 2026-02-21T23:06:59.334 INFO:teuthology.orchestra.run.trial150.stderr:2026-02-21T23:06:59.332+0000 7f0761e29780 -1 bluestore(/var/lib/ceph/osd/ceph-9) _read_fsid unparsable uuid 2026-02-21T23:06:59.659 DEBUG:teuthology.orchestra.run.trial150:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-9 2026-02-21T23:06:59.725 INFO:tasks.ceph:Reading keys from all nodes... 2026-02-21T23:06:59.726 DEBUG:teuthology.orchestra.run.trial150:> set -ex 2026-02-21T23:06:59.726 DEBUG:teuthology.orchestra.run.trial150:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2026-02-21T23:06:59.791 DEBUG:teuthology.orchestra.run.trial150:> set -ex 2026-02-21T23:06:59.791 DEBUG:teuthology.orchestra.run.trial150:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2026-02-21T23:06:59.857 DEBUG:teuthology.orchestra.run.trial150:> set -ex 2026-02-21T23:06:59.857 DEBUG:teuthology.orchestra.run.trial150:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2026-02-21T23:06:59.919 DEBUG:teuthology.orchestra.run.trial150:> set -ex 2026-02-21T23:06:59.920 DEBUG:teuthology.orchestra.run.trial150:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2026-02-21T23:06:59.983 DEBUG:teuthology.orchestra.run.trial150:> set -ex 2026-02-21T23:06:59.983 DEBUG:teuthology.orchestra.run.trial150:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2026-02-21T23:07:00.056 DEBUG:teuthology.orchestra.run.trial150:> set -ex 2026-02-21T23:07:00.056 DEBUG:teuthology.orchestra.run.trial150:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2026-02-21T23:07:00.130 DEBUG:teuthology.orchestra.run.trial150:> set -ex 2026-02-21T23:07:00.130 DEBUG:teuthology.orchestra.run.trial150:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2026-02-21T23:07:00.197 DEBUG:teuthology.orchestra.run.trial150:> set -ex 2026-02-21T23:07:00.197 DEBUG:teuthology.orchestra.run.trial150:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2026-02-21T23:07:00.261 DEBUG:teuthology.orchestra.run.trial150:> set -ex 2026-02-21T23:07:00.262 DEBUG:teuthology.orchestra.run.trial150:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2026-02-21T23:07:00.326 DEBUG:teuthology.orchestra.run.trial150:> set -ex 2026-02-21T23:07:00.326 DEBUG:teuthology.orchestra.run.trial150:> sudo dd if=/var/lib/ceph/osd/ceph-8/keyring of=/dev/stdout 2026-02-21T23:07:00.389 DEBUG:teuthology.orchestra.run.trial150:> set -ex 2026-02-21T23:07:00.390 DEBUG:teuthology.orchestra.run.trial150:> sudo dd if=/var/lib/ceph/osd/ceph-9/keyring of=/dev/stdout 2026-02-21T23:07:00.452 DEBUG:teuthology.orchestra.run.trial150:> set -ex 2026-02-21T23:07:00.452 DEBUG:teuthology.orchestra.run.trial150:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2026-02-21T23:07:00.509 INFO:tasks.ceph:Adding keys to all mons... 2026-02-21T23:07:00.509 DEBUG:teuthology.orchestra.run.trial150:> sudo tee -a /etc/ceph/ceph.keyring 2026-02-21T23:07:00.571 INFO:teuthology.orchestra.run.trial150.stdout:[mgr.x] 2026-02-21T23:07:00.571 INFO:teuthology.orchestra.run.trial150.stdout: key = AQCLOpppQc25EhAAPiizaVr7gtWaZIYjQBok4Q== 2026-02-21T23:07:00.572 INFO:teuthology.orchestra.run.trial150.stdout:[osd.0] 2026-02-21T23:07:00.572 INFO:teuthology.orchestra.run.trial150.stdout: key = AQCPOpppsx5HCxAAxqCNAgVXnhgiHb1qSAM3+Q== 2026-02-21T23:07:00.572 INFO:teuthology.orchestra.run.trial150.stdout:[osd.1] 2026-02-21T23:07:00.572 INFO:teuthology.orchestra.run.trial150.stdout: key = AQCPOpppE7HpJhAA+3ZUH9sXIsQp8DBBhLLrjA== 2026-02-21T23:07:00.572 INFO:teuthology.orchestra.run.trial150.stdout:[osd.2] 2026-02-21T23:07:00.572 INFO:teuthology.orchestra.run.trial150.stdout: key = AQCQOpppMnRJBxAAga7/0f3xpJ7eymZ7C4txAw== 2026-02-21T23:07:00.572 INFO:teuthology.orchestra.run.trial150.stdout:[osd.3] 2026-02-21T23:07:00.572 INFO:teuthology.orchestra.run.trial150.stdout: key = AQCQOpppd2fgIhAAc8fPSQh6dTpryCfSMMOxSA== 2026-02-21T23:07:00.573 INFO:teuthology.orchestra.run.trial150.stdout:[osd.4] 2026-02-21T23:07:00.573 INFO:teuthology.orchestra.run.trial150.stdout: key = AQCROpppmRY5AhAADegNXRE3JmDECbj6JBlGtg== 2026-02-21T23:07:00.573 INFO:teuthology.orchestra.run.trial150.stdout:[osd.5] 2026-02-21T23:07:00.573 INFO:teuthology.orchestra.run.trial150.stdout: key = AQCROpppICxxHRAA6zIr0xzfdieAhz2sv7hXfg== 2026-02-21T23:07:00.573 INFO:teuthology.orchestra.run.trial150.stdout:[osd.6] 2026-02-21T23:07:00.573 INFO:teuthology.orchestra.run.trial150.stdout: key = AQCROpppieJYOBAAwoMyIXv8qfzotAS+YWo9Dw== 2026-02-21T23:07:00.574 INFO:teuthology.orchestra.run.trial150.stdout:[osd.7] 2026-02-21T23:07:00.574 INFO:teuthology.orchestra.run.trial150.stdout: key = AQCSOpppr9R+GBAAurEQEywfn0B/flHkvBx3Yw== 2026-02-21T23:07:00.574 INFO:teuthology.orchestra.run.trial150.stdout:[osd.8] 2026-02-21T23:07:00.574 INFO:teuthology.orchestra.run.trial150.stdout: key = AQCSOpppVt2EMxAATPYzxwahlfG3Bz22VhObEA== 2026-02-21T23:07:00.574 INFO:teuthology.orchestra.run.trial150.stdout:[osd.9] 2026-02-21T23:07:00.574 INFO:teuthology.orchestra.run.trial150.stdout: key = AQCTOpppkPLeExAAl5p2wwLQWpizivjzCc6yZQ== 2026-02-21T23:07:00.574 INFO:teuthology.orchestra.run.trial150.stdout:[client.0] 2026-02-21T23:07:00.574 INFO:teuthology.orchestra.run.trial150.stdout: key = AQCLOpppgv9TFxAAmWz0XqQiNB5gKTrlLC3aaw== 2026-02-21T23:07:00.576 DEBUG:teuthology.orchestra.run.trial150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2026-02-21T23:07:00.654 DEBUG:teuthology.orchestra.run.trial150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-21T23:07:00.734 DEBUG:teuthology.orchestra.run.trial150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-21T23:07:00.813 DEBUG:teuthology.orchestra.run.trial150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-21T23:07:00.894 DEBUG:teuthology.orchestra.run.trial150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-21T23:07:00.972 DEBUG:teuthology.orchestra.run.trial150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-21T23:07:01.050 DEBUG:teuthology.orchestra.run.trial150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-21T23:07:01.129 DEBUG:teuthology.orchestra.run.trial150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-21T23:07:01.208 DEBUG:teuthology.orchestra.run.trial150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-21T23:07:01.287 DEBUG:teuthology.orchestra.run.trial150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.8 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-21T23:07:01.365 DEBUG:teuthology.orchestra.run.trial150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.9 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-21T23:07:01.443 DEBUG:teuthology.orchestra.run.trial150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2026-02-21T23:07:01.524 INFO:tasks.ceph:Running mkfs on mon nodes... 2026-02-21T23:07:01.524 DEBUG:teuthology.orchestra.run.trial150:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2026-02-21T23:07:01.588 DEBUG:teuthology.orchestra.run.trial150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2026-02-21T23:07:01.668 DEBUG:teuthology.orchestra.run.trial150:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2026-02-21T23:07:01.732 DEBUG:teuthology.orchestra.run.trial150:> rm -- /home/ubuntu/cephtest/ceph.monmap 2026-02-21T23:07:01.789 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2026-02-21T23:07:01.789 INFO:tasks.ceph.mon.a:Restarting daemon 2026-02-21T23:07:01.789 DEBUG:teuthology.orchestra.run.trial150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2026-02-21T23:07:01.831 INFO:tasks.ceph.mon.a:Started 2026-02-21T23:07:01.832 DEBUG:teuthology.orchestra.run.trial150:> sudo ceph config set mgr mgr_pool false --force 2026-02-21T23:07:01.938 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2026-02-21T23:07:01.938 INFO:tasks.ceph.mgr.x:Restarting daemon 2026-02-21T23:07:01.938 DEBUG:teuthology.orchestra.run.trial150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2026-02-21T23:07:01.948 INFO:tasks.ceph.mgr.x:Started 2026-02-21T23:07:01.948 DEBUG:tasks.ceph:set 0 configs 2026-02-21T23:07:01.949 DEBUG:teuthology.orchestra.run.trial150:> sudo ceph --cluster ceph config dump 2026-02-21T23:07:02.013 INFO:tasks.ceph.mgr.x.trial150.stderr:2026-02-21T23:07:02.012+0000 7fab92fc0fc0 -1 mgr[py] Loading module stats in main interpreter 2026-02-21T23:07:02.046 INFO:tasks.ceph.mgr.x.trial150.stderr:2026-02-21T23:07:02.046+0000 7fab92fc0fc0 -1 mgr[py] Loading module dashboard in main interpreter 2026-02-21T23:07:02.058 INFO:teuthology.orchestra.run.trial150.stdout:WHO MASK LEVEL OPTION VALUE RO 2026-02-21T23:07:02.058 INFO:teuthology.orchestra.run.trial150.stdout:mgr dev mgr_pool false * 2026-02-21T23:07:02.067 INFO:tasks.ceph:Setting crush tunables to default 2026-02-21T23:07:02.067 DEBUG:teuthology.orchestra.run.trial150:> sudo ceph --cluster ceph osd crush tunables default 2026-02-21T23:07:02.167 INFO:teuthology.orchestra.run.trial150.stderr:adjusted tunables profile to default 2026-02-21T23:07:02.175 INFO:tasks.ceph:check_enable_crimson: False 2026-02-21T23:07:02.175 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2026-02-21T23:07:02.176 DEBUG:teuthology.orchestra.run.trial150:> set -ex 2026-02-21T23:07:02.176 DEBUG:teuthology.orchestra.run.trial150:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2026-02-21T23:07:02.201 DEBUG:teuthology.orchestra.run.trial150:> set -ex 2026-02-21T23:07:02.202 DEBUG:teuthology.orchestra.run.trial150:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2026-02-21T23:07:02.266 DEBUG:teuthology.orchestra.run.trial150:> set -ex 2026-02-21T23:07:02.266 DEBUG:teuthology.orchestra.run.trial150:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2026-02-21T23:07:02.331 DEBUG:teuthology.orchestra.run.trial150:> set -ex 2026-02-21T23:07:02.332 DEBUG:teuthology.orchestra.run.trial150:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2026-02-21T23:07:02.339 INFO:tasks.ceph.mgr.x.trial150.stderr:2026-02-21T23:07:02.338+0000 7fab92fc0fc0 -1 mgr[py] Loading module mirroring in main interpreter 2026-02-21T23:07:02.347 INFO:tasks.ceph.mgr.x.trial150.stderr:2026-02-21T23:07:02.346+0000 7fab92fc0fc0 -1 mgr[py] Loading module snap_schedule in main interpreter 2026-02-21T23:07:02.353 INFO:tasks.ceph.mgr.x.trial150.stderr:2026-02-21T23:07:02.352+0000 7fab92fc0fc0 -1 mgr[py] Loading module prometheus in main interpreter 2026-02-21T23:07:02.359 INFO:tasks.ceph.mgr.x.trial150.stderr:2026-02-21T23:07:02.358+0000 7fab92fc0fc0 -1 mgr[py] Loading module balancer in main interpreter 2026-02-21T23:07:02.363 DEBUG:teuthology.orchestra.run.trial150:> set -ex 2026-02-21T23:07:02.363 DEBUG:teuthology.orchestra.run.trial150:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2026-02-21T23:07:02.364 INFO:tasks.ceph.mgr.x.trial150.stderr:2026-02-21T23:07:02.363+0000 7fab92fc0fc0 -1 mgr[py] Loading module rbd_support in main interpreter 2026-02-21T23:07:02.376 INFO:tasks.ceph.mgr.x.trial150.stderr:2026-02-21T23:07:02.375+0000 7fab92fc0fc0 -1 mgr[py] Loading module smb in main interpreter 2026-02-21T23:07:02.387 DEBUG:teuthology.orchestra.run.trial150:> set -ex 2026-02-21T23:07:02.388 DEBUG:teuthology.orchestra.run.trial150:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2026-02-21T23:07:02.397 INFO:tasks.ceph.mgr.x.trial150.stderr:2026-02-21T23:07:02.396+0000 7fab92fc0fc0 -1 mgr[py] Loading module devicehealth in main interpreter 2026-02-21T23:07:02.400 INFO:tasks.ceph.mgr.x.trial150.stderr:2026-02-21T23:07:02.399+0000 7fab92fc0fc0 -1 mgr[py] Loading module alerts in main interpreter 2026-02-21T23:07:02.401 INFO:tasks.ceph.mgr.x.trial150.stderr:2026-02-21T23:07:02.401+0000 7fab92fc0fc0 -1 mgr[py] Loading module telemetry in main interpreter 2026-02-21T23:07:02.407 INFO:tasks.ceph.mgr.x.trial150.stderr:2026-02-21T23:07:02.406+0000 7fab92fc0fc0 -1 mgr[py] Loading module diskprediction_local in main interpreter 2026-02-21T23:07:02.422 DEBUG:teuthology.orchestra.run.trial150:> set -ex 2026-02-21T23:07:02.422 DEBUG:teuthology.orchestra.run.trial150:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2026-02-21T23:07:02.442 INFO:tasks.ceph.mgr.x.trial150.stderr:2026-02-21T23:07:02.442+0000 7fab92fc0fc0 -1 mgr[py] Loading module status in main interpreter 2026-02-21T23:07:02.445 INFO:tasks.ceph.mgr.x.trial150.stderr:2026-02-21T23:07:02.443+0000 7fab92fc0fc0 -1 mgr[py] Loading module rgw in main interpreter 2026-02-21T23:07:02.447 INFO:tasks.ceph.mgr.x.trial150.stderr:2026-02-21T23:07:02.447+0000 7fab92fc0fc0 -1 mgr[py] Loading module insights in main interpreter 2026-02-21T23:07:02.449 INFO:tasks.ceph.mgr.x.trial150.stderr:2026-02-21T23:07:02.449+0000 7fab92fc0fc0 -1 mgr[py] Loading module iostat in main interpreter 2026-02-21T23:07:02.450 INFO:tasks.ceph.mgr.x.trial150.stderr:2026-02-21T23:07:02.449+0000 7fab92fc0fc0 -1 mgr[py] Loading module pg_autoscaler in main interpreter 2026-02-21T23:07:02.466 DEBUG:teuthology.orchestra.run.trial150:> set -ex 2026-02-21T23:07:02.466 DEBUG:teuthology.orchestra.run.trial150:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2026-02-21T23:07:02.467 INFO:tasks.ceph.mgr.x.trial150.stderr:2026-02-21T23:07:02.465+0000 7fab92fc0fc0 -1 mgr[py] Loading module osd_perf_query in main interpreter 2026-02-21T23:07:02.467 INFO:tasks.ceph.mgr.x.trial150.stderr:2026-02-21T23:07:02.466+0000 7fab92fc0fc0 -1 mgr[py] Loading module crash in main interpreter 2026-02-21T23:07:02.469 INFO:tasks.ceph.mgr.x.trial150.stderr:2026-02-21T23:07:02.468+0000 7fab92fc0fc0 -1 mgr[py] Loading module progress in main interpreter 2026-02-21T23:07:02.471 INFO:tasks.ceph.mgr.x.trial150.stderr:2026-02-21T23:07:02.470+0000 7fab92fc0fc0 -1 mgr[py] Loading module rook in main interpreter 2026-02-21T23:07:02.492 DEBUG:teuthology.orchestra.run.trial150:> set -ex 2026-02-21T23:07:02.492 DEBUG:teuthology.orchestra.run.trial150:> sudo dd if=/var/lib/ceph/osd/ceph-8/fsid of=/dev/stdout 2026-02-21T23:07:02.557 DEBUG:teuthology.orchestra.run.trial150:> set -ex 2026-02-21T23:07:02.557 DEBUG:teuthology.orchestra.run.trial150:> sudo dd if=/var/lib/ceph/osd/ceph-9/fsid of=/dev/stdout 2026-02-21T23:07:02.582 INFO:tasks.ceph.mgr.x.trial150.stderr:2026-02-21T23:07:02.581+0000 7fab92fc0fc0 -1 mgr[py] Loading module nfs in main interpreter 2026-02-21T23:07:02.582 INFO:tasks.ceph.mgr.x.trial150.stderr:2026-02-21T23:07:02.581+0000 7fab92fc0fc0 -1 mgr[py] Loading module telegraf in main interpreter 2026-02-21T23:07:02.585 INFO:tasks.ceph.mgr.x.trial150.stderr:2026-02-21T23:07:02.583+0000 7fab92fc0fc0 -1 mgr[py] Loading module selftest in main interpreter 2026-02-21T23:07:02.587 INFO:tasks.ceph.mgr.x.trial150.stderr:2026-02-21T23:07:02.586+0000 7fab92fc0fc0 -1 mgr[py] Loading module test_orchestrator in main interpreter 2026-02-21T23:07:02.589 INFO:tasks.ceph.mgr.x.trial150.stderr:2026-02-21T23:07:02.588+0000 7fab92fc0fc0 -1 mgr[py] Loading module osd_support in main interpreter 2026-02-21T23:07:02.589 INFO:tasks.ceph.mgr.x.trial150.stderr:2026-02-21T23:07:02.588+0000 7fab92fc0fc0 -1 mgr[py] Loading module orchestrator in main interpreter 2026-02-21T23:07:02.589 INFO:tasks.ceph.mgr.x.trial150.stderr:2026-02-21T23:07:02.589+0000 7fab92fc0fc0 -1 mgr[py] Loading module cephadm in main interpreter 2026-02-21T23:07:02.606 DEBUG:teuthology.orchestra.run.trial150:> sudo ceph --cluster ceph osd new c4ec8e56-7376-4e6a-84af-d03afaa3dbee 0 2026-02-21T23:07:02.747 INFO:teuthology.orchestra.run.trial150.stdout:0 2026-02-21T23:07:02.755 DEBUG:teuthology.orchestra.run.trial150:> sudo ceph --cluster ceph osd new 389ddd9e-6beb-48ce-8ee1-574db38a866c 1 2026-02-21T23:07:02.773 INFO:tasks.ceph.mgr.x.trial150.stderr:2026-02-21T23:07:02.772+0000 7fab92fc0fc0 -1 mgr[py] Loading module volumes in main interpreter 2026-02-21T23:07:02.797 INFO:tasks.ceph.mgr.x.trial150.stderr:2026-02-21T23:07:02.797+0000 7fab92fc0fc0 -1 mgr[py] Loading module k8sevents in main interpreter 2026-02-21T23:07:02.801 INFO:tasks.ceph.mgr.x.trial150.stderr:2026-02-21T23:07:02.800+0000 7fab92fc0fc0 -1 mgr[py] Loading module mds_autoscaler in main interpreter 2026-02-21T23:07:02.801 INFO:tasks.ceph.mgr.x.trial150.stderr:2026-02-21T23:07:02.801+0000 7fab92fc0fc0 -1 mgr[py] Loading module localpool in main interpreter 2026-02-21T23:07:02.802 INFO:tasks.ceph.mgr.x.trial150.stderr:2026-02-21T23:07:02.801+0000 7fab92fc0fc0 -1 mgr[py] Loading module influx in main interpreter 2026-02-21T23:07:02.857 INFO:teuthology.orchestra.run.trial150.stdout:1 2026-02-21T23:07:02.865 DEBUG:teuthology.orchestra.run.trial150:> sudo ceph --cluster ceph osd new 718e49c5-1294-46b1-b1b4-fa3edda8564c 2 2026-02-21T23:07:03.006 INFO:teuthology.orchestra.run.trial150.stdout:2 2026-02-21T23:07:03.013 DEBUG:teuthology.orchestra.run.trial150:> sudo ceph --cluster ceph osd new 9a09d418-342f-4db0-a6e1-700059b6f0f4 3 2026-02-21T23:07:03.114 INFO:teuthology.orchestra.run.trial150.stdout:3 2026-02-21T23:07:03.121 DEBUG:teuthology.orchestra.run.trial150:> sudo ceph --cluster ceph osd new e19f379d-43d8-4536-ac93-ad0d30c98828 4 2026-02-21T23:07:03.224 INFO:teuthology.orchestra.run.trial150.stdout:4 2026-02-21T23:07:03.231 DEBUG:teuthology.orchestra.run.trial150:> sudo ceph --cluster ceph osd new 5d7c9240-c8e9-40d3-835b-ded2969b7c24 5 2026-02-21T23:07:03.361 INFO:teuthology.orchestra.run.trial150.stdout:5 2026-02-21T23:07:03.368 DEBUG:teuthology.orchestra.run.trial150:> sudo ceph --cluster ceph osd new 0978d144-b749-4b5e-9c84-9a6c2f1bc3dc 6 2026-02-21T23:07:03.474 INFO:teuthology.orchestra.run.trial150.stdout:6 2026-02-21T23:07:03.482 DEBUG:teuthology.orchestra.run.trial150:> sudo ceph --cluster ceph osd new 06c1e800-e425-4898-9bbf-fbf1b196d33e 7 2026-02-21T23:07:03.587 INFO:teuthology.orchestra.run.trial150.stdout:7 2026-02-21T23:07:03.595 DEBUG:teuthology.orchestra.run.trial150:> sudo ceph --cluster ceph osd new 52f40cc0-0808-4b7d-a711-e77d7d19ff58 8 2026-02-21T23:07:03.695 INFO:teuthology.orchestra.run.trial150.stdout:8 2026-02-21T23:07:03.702 DEBUG:teuthology.orchestra.run.trial150:> sudo ceph --cluster ceph osd new 74e2070c-75ed-4e1b-b25a-86614f45a1fb 9 2026-02-21T23:07:03.804 INFO:teuthology.orchestra.run.trial150.stdout:9 2026-02-21T23:07:03.811 INFO:tasks.ceph.osd.0:Restarting daemon 2026-02-21T23:07:03.812 DEBUG:teuthology.orchestra.run.trial150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2026-02-21T23:07:03.813 INFO:tasks.ceph.osd.0:Started 2026-02-21T23:07:03.813 INFO:tasks.ceph.osd.1:Restarting daemon 2026-02-21T23:07:03.813 DEBUG:teuthology.orchestra.run.trial150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2026-02-21T23:07:03.815 INFO:tasks.ceph.osd.1:Started 2026-02-21T23:07:03.815 INFO:tasks.ceph.osd.2:Restarting daemon 2026-02-21T23:07:03.815 DEBUG:teuthology.orchestra.run.trial150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2026-02-21T23:07:03.816 INFO:tasks.ceph.osd.2:Started 2026-02-21T23:07:03.816 INFO:tasks.ceph.osd.3:Restarting daemon 2026-02-21T23:07:03.816 DEBUG:teuthology.orchestra.run.trial150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2026-02-21T23:07:03.816 INFO:tasks.ceph.mgr.x.trial150.stderr:2026-02-21T23:07:03.815+0000 7faabb8eb640 -1 mgr.server handle_report got status from non-daemon mon.a 2026-02-21T23:07:03.818 INFO:tasks.ceph.osd.3:Started 2026-02-21T23:07:03.818 INFO:tasks.ceph.osd.4:Restarting daemon 2026-02-21T23:07:03.818 DEBUG:teuthology.orchestra.run.trial150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2026-02-21T23:07:03.819 INFO:tasks.ceph.osd.4:Started 2026-02-21T23:07:03.819 INFO:tasks.ceph.osd.5:Restarting daemon 2026-02-21T23:07:03.819 DEBUG:teuthology.orchestra.run.trial150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2026-02-21T23:07:03.820 INFO:tasks.ceph.osd.5:Started 2026-02-21T23:07:03.820 INFO:tasks.ceph.osd.6:Restarting daemon 2026-02-21T23:07:03.821 DEBUG:teuthology.orchestra.run.trial150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2026-02-21T23:07:03.822 INFO:tasks.ceph.osd.6:Started 2026-02-21T23:07:03.822 INFO:tasks.ceph.osd.7:Restarting daemon 2026-02-21T23:07:03.822 DEBUG:teuthology.orchestra.run.trial150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2026-02-21T23:07:03.823 INFO:tasks.ceph.osd.7:Started 2026-02-21T23:07:03.824 INFO:tasks.ceph.osd.8:Restarting daemon 2026-02-21T23:07:03.824 DEBUG:teuthology.orchestra.run.trial150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 8 2026-02-21T23:07:03.825 INFO:tasks.ceph.osd.8:Started 2026-02-21T23:07:03.825 INFO:tasks.ceph.osd.9:Restarting daemon 2026-02-21T23:07:03.825 DEBUG:teuthology.orchestra.run.trial150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 9 2026-02-21T23:07:03.826 INFO:tasks.ceph.osd.9:Started 2026-02-21T23:07:03.827 DEBUG:teuthology.orchestra.run.trial150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2026-02-21T23:07:04.014 INFO:teuthology.orchestra.run.trial150.stdout: 2026-02-21T23:07:04.014 INFO:teuthology.orchestra.run.trial150.stdout:{"epoch":12,"fsid":"f77161f3-2466-452f-b0a1-014035b869f9","created":"2026-02-21T23:07:01.892470+0000","modified":"2026-02-21T23:07:03.803679+0000","last_up_change":"0.000000","last_in_change":"2026-02-21T23:07:03.803679+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":10,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"tentacle","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"c4ec8e56-7376-4e6a-84af-d03afaa3dbee","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"389ddd9e-6beb-48ce-8ee1-574db38a866c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"718e49c5-1294-46b1-b1b4-fa3edda8564c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"9a09d418-342f-4db0-a6e1-700059b6f0f4","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"e19f379d-43d8-4536-ac93-ad0d30c98828","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"5d7c9240-c8e9-40d3-835b-ded2969b7c24","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"0978d144-b749-4b5e-9c84-9a6c2f1bc3dc","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"06c1e800-e425-4898-9bbf-fbf1b196d33e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":8,"uuid":"52f40cc0-0808-4b7d-a711-e77d7d19ff58","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":9,"uuid":"74e2070c-75ed-4e1b-b25a-86614f45a1fb","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"isa","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2026-02-21T23:07:04.024 INFO:tasks.ceph.ceph_manager.ceph:[] 2026-02-21T23:07:04.024 INFO:tasks.ceph:Waiting for OSDs to come up 2026-02-21T23:07:04.075 INFO:tasks.ceph.osd.4.trial150.stderr:2026-02-21T23:07:04.074+0000 7fea09808780 -1 Falling back to public interface 2026-02-21T23:07:04.076 INFO:tasks.ceph.osd.1.trial150.stderr:2026-02-21T23:07:04.075+0000 7ffafec31780 -1 Falling back to public interface 2026-02-21T23:07:04.082 INFO:tasks.ceph.osd.6.trial150.stderr:2026-02-21T23:07:04.081+0000 7f527b42f780 -1 Falling back to public interface 2026-02-21T23:07:04.090 INFO:tasks.ceph.osd.2.trial150.stderr:2026-02-21T23:07:04.089+0000 7fee98a39780 -1 Falling back to public interface 2026-02-21T23:07:04.090 INFO:tasks.ceph.osd.7.trial150.stderr:2026-02-21T23:07:04.089+0000 7f67d4238780 -1 Falling back to public interface 2026-02-21T23:07:04.091 INFO:tasks.ceph.osd.9.trial150.stderr:2026-02-21T23:07:04.090+0000 7f11cd814780 -1 Falling back to public interface 2026-02-21T23:07:04.091 INFO:tasks.ceph.osd.3.trial150.stderr:2026-02-21T23:07:04.090+0000 7f2189633780 -1 Falling back to public interface 2026-02-21T23:07:04.102 INFO:tasks.ceph.osd.0.trial150.stderr:2026-02-21T23:07:04.101+0000 7f64cffdf780 -1 Falling back to public interface 2026-02-21T23:07:04.102 INFO:tasks.ceph.osd.8.trial150.stderr:2026-02-21T23:07:04.101+0000 7f24ef40b780 -1 Falling back to public interface 2026-02-21T23:07:04.118 INFO:tasks.ceph.osd.5.trial150.stderr:2026-02-21T23:07:04.117+0000 7f3b3a53d780 -1 Falling back to public interface 2026-02-21T23:07:04.514 INFO:tasks.ceph.osd.7.trial150.stderr:2026-02-21T23:07:04.513+0000 7f67d4238780 -1 osd.7 0 log_to_monitors true 2026-02-21T23:07:04.514 INFO:tasks.ceph.osd.1.trial150.stderr:2026-02-21T23:07:04.513+0000 7ffafec31780 -1 osd.1 0 log_to_monitors true 2026-02-21T23:07:04.518 INFO:tasks.ceph.osd.4.trial150.stderr:2026-02-21T23:07:04.517+0000 7fea09808780 -1 osd.4 0 log_to_monitors true 2026-02-21T23:07:04.523 INFO:tasks.ceph.osd.6.trial150.stderr:2026-02-21T23:07:04.522+0000 7f527b42f780 -1 osd.6 0 log_to_monitors true 2026-02-21T23:07:04.523 INFO:tasks.ceph.osd.2.trial150.stderr:2026-02-21T23:07:04.522+0000 7fee98a39780 -1 osd.2 0 log_to_monitors true 2026-02-21T23:07:04.527 INFO:tasks.ceph.osd.3.trial150.stderr:2026-02-21T23:07:04.526+0000 7f2189633780 -1 osd.3 0 log_to_monitors true 2026-02-21T23:07:04.538 INFO:tasks.ceph.osd.0.trial150.stderr:2026-02-21T23:07:04.537+0000 7f64cffdf780 -1 osd.0 0 log_to_monitors true 2026-02-21T23:07:04.540 INFO:tasks.ceph.osd.9.trial150.stderr:2026-02-21T23:07:04.539+0000 7f11cd814780 -1 osd.9 0 log_to_monitors true 2026-02-21T23:07:04.555 INFO:tasks.ceph.osd.8.trial150.stderr:2026-02-21T23:07:04.554+0000 7f24ef40b780 -1 osd.8 0 log_to_monitors true 2026-02-21T23:07:04.564 INFO:tasks.ceph.osd.5.trial150.stderr:2026-02-21T23:07:04.564+0000 7f3b3a53d780 -1 osd.5 0 log_to_monitors true 2026-02-21T23:07:05.029 DEBUG:teuthology.orchestra.run.trial150:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2026-02-21T23:07:05.199 INFO:teuthology.misc.health.trial150.stdout: 2026-02-21T23:07:05.199 INFO:teuthology.misc.health.trial150.stdout:{"epoch":13,"fsid":"f77161f3-2466-452f-b0a1-014035b869f9","created":"2026-02-21T23:07:01.892470+0000","modified":"2026-02-21T23:07:04.897328+0000","last_up_change":"0.000000","last_in_change":"2026-02-21T23:07:03.803679+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":3,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":10,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"tentacle","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"c4ec8e56-7376-4e6a-84af-d03afaa3dbee","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"389ddd9e-6beb-48ce-8ee1-574db38a866c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"718e49c5-1294-46b1-b1b4-fa3edda8564c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"9a09d418-342f-4db0-a6e1-700059b6f0f4","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"e19f379d-43d8-4536-ac93-ad0d30c98828","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"5d7c9240-c8e9-40d3-835b-ded2969b7c24","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"0978d144-b749-4b5e-9c84-9a6c2f1bc3dc","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"06c1e800-e425-4898-9bbf-fbf1b196d33e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":8,"uuid":"52f40cc0-0808-4b7d-a711-e77d7d19ff58","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":9,"uuid":"74e2070c-75ed-4e1b-b25a-86614f45a1fb","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"isa","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2026-02-21T23:07:05.207 DEBUG:teuthology.misc:0 of 10 OSDs are up 2026-02-21T23:07:05.907 INFO:tasks.ceph.osd.1.trial150.stderr:2026-02-21T23:07:05.906+0000 7ffaf9bf7640 -1 osd.1 0 waiting for initial osdmap 2026-02-21T23:07:05.907 INFO:tasks.ceph.osd.2.trial150.stderr:2026-02-21T23:07:05.906+0000 7fee939f7640 -1 osd.2 0 waiting for initial osdmap 2026-02-21T23:07:05.908 INFO:tasks.ceph.osd.4.trial150.stderr:2026-02-21T23:07:05.906+0000 7fea04bf7640 -1 osd.4 0 waiting for initial osdmap 2026-02-21T23:07:05.908 INFO:tasks.ceph.osd.3.trial150.stderr:2026-02-21T23:07:05.906+0000 7f21845f7640 -1 osd.3 0 waiting for initial osdmap 2026-02-21T23:07:05.908 INFO:tasks.ceph.osd.9.trial150.stderr:2026-02-21T23:07:05.906+0000 7f11c8bf7640 -1 osd.9 0 waiting for initial osdmap 2026-02-21T23:07:05.909 INFO:tasks.ceph.osd.6.trial150.stderr:2026-02-21T23:07:05.906+0000 7f52763f7640 -1 osd.6 0 waiting for initial osdmap 2026-02-21T23:07:05.909 INFO:tasks.ceph.osd.5.trial150.stderr:2026-02-21T23:07:05.907+0000 7f3b361f7640 -1 osd.5 0 waiting for initial osdmap 2026-02-21T23:07:05.909 INFO:tasks.ceph.osd.0.trial150.stderr:2026-02-21T23:07:05.907+0000 7f64cb5f7640 -1 osd.0 0 waiting for initial osdmap 2026-02-21T23:07:05.910 INFO:tasks.ceph.osd.8.trial150.stderr:2026-02-21T23:07:05.907+0000 7f24ea7f7640 -1 osd.8 0 waiting for initial osdmap 2026-02-21T23:07:05.910 INFO:tasks.ceph.osd.7.trial150.stderr:2026-02-21T23:07:05.907+0000 7f67cf5f7640 -1 osd.7 0 waiting for initial osdmap 2026-02-21T23:07:05.910 INFO:tasks.ceph.osd.3.trial150.stderr:2026-02-21T23:07:05.908+0000 7f21805ef640 -1 osd.3 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2026-02-21T23:07:05.911 INFO:tasks.ceph.osd.1.trial150.stderr:2026-02-21T23:07:05.909+0000 7ffaf5bef640 -1 osd.1 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2026-02-21T23:07:05.911 INFO:tasks.ceph.osd.4.trial150.stderr:2026-02-21T23:07:05.909+0000 7fe9fcbe7640 -1 osd.4 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2026-02-21T23:07:05.911 INFO:tasks.ceph.osd.6.trial150.stderr:2026-02-21T23:07:05.909+0000 7f52723ef640 -1 osd.6 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2026-02-21T23:07:05.912 INFO:tasks.ceph.osd.2.trial150.stderr:2026-02-21T23:07:05.909+0000 7fee8f9ef640 -1 osd.2 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2026-02-21T23:07:05.912 INFO:tasks.ceph.osd.9.trial150.stderr:2026-02-21T23:07:05.909+0000 7f11c0be7640 -1 osd.9 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2026-02-21T23:07:05.913 INFO:tasks.ceph.osd.0.trial150.stderr:2026-02-21T23:07:05.909+0000 7f64c352a640 -1 osd.0 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2026-02-21T23:07:05.913 INFO:tasks.ceph.osd.5.trial150.stderr:2026-02-21T23:07:05.909+0000 7f3b2e12a640 -1 osd.5 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2026-02-21T23:07:05.913 INFO:tasks.ceph.osd.8.trial150.stderr:2026-02-21T23:07:05.909+0000 7f24e6732640 -1 osd.8 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2026-02-21T23:07:05.913 INFO:tasks.ceph.osd.7.trial150.stderr:2026-02-21T23:07:05.910+0000 7f67c75e7640 -1 osd.7 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2026-02-21T23:07:12.213 DEBUG:teuthology.orchestra.run.trial150:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2026-02-21T23:07:12.388 INFO:teuthology.misc.health.trial150.stdout: 2026-02-21T23:07:12.389 INFO:teuthology.misc.health.trial150.stdout:{"epoch":16,"fsid":"f77161f3-2466-452f-b0a1-014035b869f9","created":"2026-02-21T23:07:01.892470+0000","modified":"2026-02-21T23:07:07.928005+0000","last_up_change":"2026-02-21T23:07:06.902989+0000","last_in_change":"2026-02-21T23:07:03.803679+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":10,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"tentacle","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"c4ec8e56-7376-4e6a-84af-d03afaa3dbee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.150:6858","nonce":1860146791},{"type":"v1","addr":"10.20.193.150:6860","nonce":1860146791}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.150:6862","nonce":1860146791},{"type":"v1","addr":"10.20.193.150:6864","nonce":1860146791}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.150:6870","nonce":1860146791},{"type":"v1","addr":"10.20.193.150:6872","nonce":1860146791}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.150:6866","nonce":1860146791},{"type":"v1","addr":"10.20.193.150:6868","nonce":1860146791}]},"public_addr":"10.20.193.150:6860/1860146791","cluster_addr":"10.20.193.150:6864/1860146791","heartbeat_back_addr":"10.20.193.150:6872/1860146791","heartbeat_front_addr":"10.20.193.150:6868/1860146791","state":["exists","up"]},{"osd":1,"uuid":"389ddd9e-6beb-48ce-8ee1-574db38a866c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.150:6810","nonce":2256230450},{"type":"v1","addr":"10.20.193.150:6811","nonce":2256230450}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.150:6812","nonce":2256230450},{"type":"v1","addr":"10.20.193.150:6813","nonce":2256230450}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.150:6816","nonce":2256230450},{"type":"v1","addr":"10.20.193.150:6817","nonce":2256230450}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.150:6814","nonce":2256230450},{"type":"v1","addr":"10.20.193.150:6815","nonce":2256230450}]},"public_addr":"10.20.193.150:6811/2256230450","cluster_addr":"10.20.193.150:6813/2256230450","heartbeat_back_addr":"10.20.193.150:6817/2256230450","heartbeat_front_addr":"10.20.193.150:6815/2256230450","state":["exists","up"]},{"osd":2,"uuid":"718e49c5-1294-46b1-b1b4-fa3edda8564c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.150:6826","nonce":3930449501},{"type":"v1","addr":"10.20.193.150:6828","nonce":3930449501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.150:6830","nonce":3930449501},{"type":"v1","addr":"10.20.193.150:6833","nonce":3930449501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.150:6844","nonce":3930449501},{"type":"v1","addr":"10.20.193.150:6847","nonce":3930449501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.150:6837","nonce":3930449501},{"type":"v1","addr":"10.20.193.150:6840","nonce":3930449501}]},"public_addr":"10.20.193.150:6828/3930449501","cluster_addr":"10.20.193.150:6833/3930449501","heartbeat_back_addr":"10.20.193.150:6847/3930449501","heartbeat_front_addr":"10.20.193.150:6840/3930449501","state":["exists","up"]},{"osd":3,"uuid":"9a09d418-342f-4db0-a6e1-700059b6f0f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.150:6834","nonce":317686896},{"type":"v1","addr":"10.20.193.150:6839","nonce":317686896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.150:6843","nonce":317686896},{"type":"v1","addr":"10.20.193.150:6848","nonce":317686896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.150:6855","nonce":317686896},{"type":"v1","addr":"10.20.193.150:6857","nonce":317686896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.150:6851","nonce":317686896},{"type":"v1","addr":"10.20.193.150:6853","nonce":317686896}]},"public_addr":"10.20.193.150:6839/317686896","cluster_addr":"10.20.193.150:6848/317686896","heartbeat_back_addr":"10.20.193.150:6857/317686896","heartbeat_front_addr":"10.20.193.150:6853/317686896","state":["exists","up"]},{"osd":4,"uuid":"e19f379d-43d8-4536-ac93-ad0d30c98828","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.150:6802","nonce":1337979806},{"type":"v1","addr":"10.20.193.150:6803","nonce":1337979806}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.150:6804","nonce":1337979806},{"type":"v1","addr":"10.20.193.150:6805","nonce":1337979806}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.150:6808","nonce":1337979806},{"type":"v1","addr":"10.20.193.150:6809","nonce":1337979806}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.150:6806","nonce":1337979806},{"type":"v1","addr":"10.20.193.150:6807","nonce":1337979806}]},"public_addr":"10.20.193.150:6803/1337979806","cluster_addr":"10.20.193.150:6805/1337979806","heartbeat_back_addr":"10.20.193.150:6809/1337979806","heartbeat_front_addr":"10.20.193.150:6807/1337979806","state":["exists","up"]},{"osd":5,"uuid":"5d7c9240-c8e9-40d3-835b-ded2969b7c24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.150:6874","nonce":1455759585},{"type":"v1","addr":"10.20.193.150:6875","nonce":1455759585}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.150:6876","nonce":1455759585},{"type":"v1","addr":"10.20.193.150:6877","nonce":1455759585}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.150:6880","nonce":1455759585},{"type":"v1","addr":"10.20.193.150:6881","nonce":1455759585}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.150:6878","nonce":1455759585},{"type":"v1","addr":"10.20.193.150:6879","nonce":1455759585}]},"public_addr":"10.20.193.150:6875/1455759585","cluster_addr":"10.20.193.150:6877/1455759585","heartbeat_back_addr":"10.20.193.150:6881/1455759585","heartbeat_front_addr":"10.20.193.150:6879/1455759585","state":["exists","up"]},{"osd":6,"uuid":"0978d144-b749-4b5e-9c84-9a6c2f1bc3dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.150:6818","nonce":1341103058},{"type":"v1","addr":"10.20.193.150:6819","nonce":1341103058}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.150:6820","nonce":1341103058},{"type":"v1","addr":"10.20.193.150:6821","nonce":1341103058}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.150:6824","nonce":1341103058},{"type":"v1","addr":"10.20.193.150:6825","nonce":1341103058}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.150:6822","nonce":1341103058},{"type":"v1","addr":"10.20.193.150:6823","nonce":1341103058}]},"public_addr":"10.20.193.150:6819/1341103058","cluster_addr":"10.20.193.150:6821/1341103058","heartbeat_back_addr":"10.20.193.150:6825/1341103058","heartbeat_front_addr":"10.20.193.150:6823/1341103058","state":["exists","up"]},{"osd":7,"uuid":"06c1e800-e425-4898-9bbf-fbf1b196d33e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.150:6827","nonce":1453052406},{"type":"v1","addr":"10.20.193.150:6829","nonce":1453052406}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.150:6831","nonce":1453052406},{"type":"v1","addr":"10.20.193.150:6835","nonce":1453052406}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.150:6845","nonce":1453052406},{"type":"v1","addr":"10.20.193.150:6849","nonce":1453052406}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.150:6838","nonce":1453052406},{"type":"v1","addr":"10.20.193.150:6842","nonce":1453052406}]},"public_addr":"10.20.193.150:6829/1453052406","cluster_addr":"10.20.193.150:6835/1453052406","heartbeat_back_addr":"10.20.193.150:6849/1453052406","heartbeat_front_addr":"10.20.193.150:6842/1453052406","state":["exists","up"]},{"osd":8,"uuid":"52f40cc0-0808-4b7d-a711-e77d7d19ff58","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.150:6859","nonce":2701237949},{"type":"v1","addr":"10.20.193.150:6861","nonce":2701237949}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.150:6863","nonce":2701237949},{"type":"v1","addr":"10.20.193.150:6865","nonce":2701237949}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.150:6871","nonce":2701237949},{"type":"v1","addr":"10.20.193.150:6873","nonce":2701237949}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.150:6867","nonce":2701237949},{"type":"v1","addr":"10.20.193.150:6869","nonce":2701237949}]},"public_addr":"10.20.193.150:6861/2701237949","cluster_addr":"10.20.193.150:6865/2701237949","heartbeat_back_addr":"10.20.193.150:6873/2701237949","heartbeat_front_addr":"10.20.193.150:6869/2701237949","state":["exists","up"]},{"osd":9,"uuid":"74e2070c-75ed-4e1b-b25a-86614f45a1fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.150:6832","nonce":2439465072},{"type":"v1","addr":"10.20.193.150:6836","nonce":2439465072}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.150:6841","nonce":2439465072},{"type":"v1","addr":"10.20.193.150:6846","nonce":2439465072}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.150:6854","nonce":2439465072},{"type":"v1","addr":"10.20.193.150:6856","nonce":2439465072}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.150:6850","nonce":2439465072},{"type":"v1","addr":"10.20.193.150:6852","nonce":2439465072}]},"public_addr":"10.20.193.150:6836/2439465072","cluster_addr":"10.20.193.150:6846/2439465072","heartbeat_back_addr":"10.20.193.150:6856/2439465072","heartbeat_front_addr":"10.20.193.150:6852/2439465072","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-21T23:07:05.499158+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-21T23:07:05.477193+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-21T23:07:05.479491+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-21T23:07:05.482361+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-21T23:07:05.561036+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-21T23:07:05.555880+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-21T23:07:05.537802+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-21T23:07:05.553574+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-21T23:07:05.535732+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-21T23:07:05.536298+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"isa","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2026-02-21T23:07:12.396 DEBUG:teuthology.misc:10 of 10 OSDs are up 2026-02-21T23:07:12.397 INFO:tasks.ceph:Creating RBD pool 2026-02-21T23:07:12.397 DEBUG:teuthology.orchestra.run.trial150:> sudo ceph --cluster ceph osd pool create rbd 8 2026-02-21T23:07:12.933 INFO:teuthology.orchestra.run.trial150.stderr:pool 'rbd' created 2026-02-21T23:07:12.942 DEBUG:teuthology.orchestra.run.trial150:> rbd --cluster ceph pool init rbd 2026-02-21T23:07:15.944 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2026-02-21T23:07:15.945 DEBUG:teuthology.orchestra.run.trial150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config log 1 --format=json 2026-02-21T23:07:15.946 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2026-02-21T23:07:16.127 INFO:teuthology.orchestra.run.trial150.stdout: 2026-02-21T23:07:16.136 INFO:teuthology.orchestra.run.trial150.stdout:[{"version":2,"timestamp":"2026-02-21T23:07:01.930345+0000","name":"","changes":[{"name":"mgr/mgr_pool","new_value":"false"}]}] 2026-02-21T23:07:16.137 INFO:tasks.ceph_manager:config epoch is 2 2026-02-21T23:07:16.137 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2026-02-21T23:07:16.137 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2026-02-21T23:07:16.137 DEBUG:teuthology.orchestra.run.trial150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2026-02-21T23:07:16.350 INFO:teuthology.orchestra.run.trial150.stdout: 2026-02-21T23:07:16.358 INFO:teuthology.orchestra.run.trial150.stdout:{"epoch":4,"flags":0,"active_gid":4101,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.150:6800","nonce":1108426932},{"type":"v1","addr":"10.20.193.150:6801","nonce":1108426932}]},"active_addr":"10.20.193.150:6801/1108426932","active_change":"2026-02-21T23:07:02.811203+0000","active_mgr_features":4544132024016699391,"available":true,"standbys":[],"modules":["iostat","nfs"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to, use commas to separate multiple","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_pg_upmap_activity":{"name":"update_pg_upmap_activity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Updates pg_upmap activity stats to be used in `balancer status detail`","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often the agent on each host will gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"The first TCP port the agent will try to bind to. Up to 1000 subsequent ports will be attempted if this port cannot be bound","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this option can facilitate debugging daemons that encounter runtime problems.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"How frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"Fraction of total system memory to divide among autotuned daemons. Converged systems hosting both Ceph and compute should set a lower value; systems with more than 10 GiB of memory per OSD may benefit from a slightly larger value so that the abundance of RAM can be fully utilized","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command persistent logging","long_desc":"","tags":[],"see_also":[]},"certificate_automated_rotation_enabled":{"name":"certificate_automated_rotation_enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"This flag controls whether cephadm automatically rotates certificates upon expiration.","long_desc":"","tags":[],"see_also":[]},"certificate_check_debug_mode":{"name":"certificate_check_debug_mode","type":"bool","level":"dev","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"FOR TESTING ONLY: This flag forces the certificate check instead of waiting for certificate_check_period.","long_desc":"","tags":[],"see_also":[]},"certificate_check_period":{"name":"certificate_check_period","type":"int","level":"advanced","flags":0,"default_value":"1","min":"0","max":"30","enum_allowed":[],"desc":"Specifies how often (in days) the certificate should be checked for validity.","long_desc":"","tags":[],"see_also":[]},"certificate_duration_days":{"name":"certificate_duration_days","type":"int","level":"advanced","flags":0,"default_value":"1095","min":"90","max":"3650","enum_allowed":[],"desc":"Specifies the duration of self-signed certificates generated and signed by the cephadm root CA","long_desc":"","tags":[],"see_also":[]},"certificate_renewal_threshold_days":{"name":"certificate_renewal_threshold_days","type":"int","level":"advanced","flags":0,"default_value":"30","min":"10","max":"90","enum_allowed":[],"desc":"Specifies the lead time in days to initiate certificate renewal before expiration.","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently Podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.28.1","min":"","max":"","enum_allowed":[],"desc":"Alertmanager container image","long_desc":"","tags":[],"see_also":[]},"container_image_alloy":{"name":"container_image_alloy","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/alloy:latest","min":"","max":"","enum_allowed":[],"desc":"Alloy container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"Elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:12.3.1","min":"","max":"","enum_allowed":[],"desc":"Grafana container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"Haproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_nginx":{"name":"container_image_nginx","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nginx:sclorg-nginx-126","min":"","max":"","enum_allowed":[],"desc":"Nginx container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.9.1","min":"","max":"","enum_allowed":[],"desc":"Node exporter container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.6","min":"","max":"","enum_allowed":[],"desc":"Nvmeof container image","long_desc":"","tags":[],"see_also":[]},"container_image_oauth2_proxy":{"name":"container_image_oauth2_proxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/oauth2-proxy/oauth2-proxy:v7.6.0","min":"","max":"","enum_allowed":[],"desc":"Oauth2 proxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v3.6.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba_metrics":{"name":"container_image_samba_metrics","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-metrics:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba metrics container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"Snmp gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"Seconds for which to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"quay.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"Seconds for which to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"Seconds for which to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"Location of dashboards to include in Grafana","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"How frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether the hardware monitoring daemon be deployed on every host?","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at the debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"Max number of daemons per service per host: guard against mistakes and runaways","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"Max number of OSDs that will be drained simultaneously when OSDs are removed","long_desc":"","tags":[],"see_also":[]},"max_parallel_osd_upgrades":{"name":"max_parallel_osd_upgrades","type":"int","level":"advanced","flags":0,"default_value":"16","min":"","max":"","enum_allowed":[],"desc":"Maximum number of OSD daemons upgraded in parallel.","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"Mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default IP address for RedFish API (OOB management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"Location of alerts to include in Prometheus Alertmanager","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry URL for login purposes. This is not the default registry.","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"Cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Customized SSH config file to distribute to managed hosts","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_count_max":{"name":"ssh_keepalive_count_max","type":"int","level":"advanced","flags":0,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"How many sequential SSH connection liveness check failures before the host is marked offline","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_interval":{"name":"ssh_keepalive_interval","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"How often SSH connections are checked for liveness","long_desc":"","tags":[],"see_also":[]},"stray_daemon_check_interval":{"name":"stray_daemon_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"How frequently cephadm should check for the presence of stray daemons","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use the cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest to ensure that all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROM_ALERT_CREDENTIAL_CACHE_TTL":{"name":"PROM_ALERT_CREDENTIAL_CACHE_TTL","type":"int","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_HOSTNAME_PER_DAEMON":{"name":"RGW_HOSTNAME_PER_DAEMON","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crypto_caller":{"name":"crypto_caller","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sso_oauth2":{"name":"sso_oauth2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":true,"error_string":"","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"healthcheck_history_max_entries":{"name":"healthcheck_history_max_entries","type":"int","level":"advanced","flags":1,"default_value":"1000","min":"","max":"","enum_allowed":[],"desc":"Maximum number of health check history entries to keep","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"prometheus_tls_secret_name":{"name":"prometheus_tls_secret_name","type":"str","level":"advanced","flags":0,"default_value":"rook-ceph-prometheus-server-tls","min":"","max":"","enum_allowed":[],"desc":"name of tls secret in k8s for prometheus","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"internal_store_backend":{"name":"internal_store_backend","type":"str","level":"dev","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"set internal store backend. for develoment and testing only","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"pause_cloning":{"name":"pause_cloning","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Pause asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"pause_purging":{"name":"pause_purging","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Pause asynchronous subvolume purge threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"tentacle":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"force_disabled_modules":{},"last_failure_osd_epoch":0,"active_clients":[{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"10.20.193.150:0","nonce":2124920851}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"10.20.193.150:0","nonce":3883754380}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"10.20.193.150:0","nonce":898543687}]}]} 2026-02-21T23:07:16.361 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2026-02-21T23:07:16.361 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2026-02-21T23:07:16.361 DEBUG:teuthology.orchestra.run.trial150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2026-02-21T23:07:16.536 INFO:teuthology.orchestra.run.trial150.stdout: 2026-02-21T23:07:16.537 INFO:teuthology.orchestra.run.trial150.stdout:{"epoch":20,"fsid":"f77161f3-2466-452f-b0a1-014035b869f9","created":"2026-02-21T23:07:01.892470+0000","modified":"2026-02-21T23:07:15.936364+0000","last_up_change":"2026-02-21T23:07:06.902989+0000","last_in_change":"2026-02-21T23:07:03.803679+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":10,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"tentacle","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2026-02-21T23:07:12.576318+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":20,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"nonprimary_shards":"{}","options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":0.89999997615814209,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1.0000001192092896,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c4ec8e56-7376-4e6a-84af-d03afaa3dbee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.150:6858","nonce":1860146791},{"type":"v1","addr":"10.20.193.150:6860","nonce":1860146791}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.150:6862","nonce":1860146791},{"type":"v1","addr":"10.20.193.150:6864","nonce":1860146791}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.150:6870","nonce":1860146791},{"type":"v1","addr":"10.20.193.150:6872","nonce":1860146791}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.150:6866","nonce":1860146791},{"type":"v1","addr":"10.20.193.150:6868","nonce":1860146791}]},"public_addr":"10.20.193.150:6860/1860146791","cluster_addr":"10.20.193.150:6864/1860146791","heartbeat_back_addr":"10.20.193.150:6872/1860146791","heartbeat_front_addr":"10.20.193.150:6868/1860146791","state":["exists","up"]},{"osd":1,"uuid":"389ddd9e-6beb-48ce-8ee1-574db38a866c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.150:6810","nonce":2256230450},{"type":"v1","addr":"10.20.193.150:6811","nonce":2256230450}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.150:6812","nonce":2256230450},{"type":"v1","addr":"10.20.193.150:6813","nonce":2256230450}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.150:6816","nonce":2256230450},{"type":"v1","addr":"10.20.193.150:6817","nonce":2256230450}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.150:6814","nonce":2256230450},{"type":"v1","addr":"10.20.193.150:6815","nonce":2256230450}]},"public_addr":"10.20.193.150:6811/2256230450","cluster_addr":"10.20.193.150:6813/2256230450","heartbeat_back_addr":"10.20.193.150:6817/2256230450","heartbeat_front_addr":"10.20.193.150:6815/2256230450","state":["exists","up"]},{"osd":2,"uuid":"718e49c5-1294-46b1-b1b4-fa3edda8564c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.150:6826","nonce":3930449501},{"type":"v1","addr":"10.20.193.150:6828","nonce":3930449501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.150:6830","nonce":3930449501},{"type":"v1","addr":"10.20.193.150:6833","nonce":3930449501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.150:6844","nonce":3930449501},{"type":"v1","addr":"10.20.193.150:6847","nonce":3930449501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.150:6837","nonce":3930449501},{"type":"v1","addr":"10.20.193.150:6840","nonce":3930449501}]},"public_addr":"10.20.193.150:6828/3930449501","cluster_addr":"10.20.193.150:6833/3930449501","heartbeat_back_addr":"10.20.193.150:6847/3930449501","heartbeat_front_addr":"10.20.193.150:6840/3930449501","state":["exists","up"]},{"osd":3,"uuid":"9a09d418-342f-4db0-a6e1-700059b6f0f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.150:6834","nonce":317686896},{"type":"v1","addr":"10.20.193.150:6839","nonce":317686896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.150:6843","nonce":317686896},{"type":"v1","addr":"10.20.193.150:6848","nonce":317686896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.150:6855","nonce":317686896},{"type":"v1","addr":"10.20.193.150:6857","nonce":317686896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.150:6851","nonce":317686896},{"type":"v1","addr":"10.20.193.150:6853","nonce":317686896}]},"public_addr":"10.20.193.150:6839/317686896","cluster_addr":"10.20.193.150:6848/317686896","heartbeat_back_addr":"10.20.193.150:6857/317686896","heartbeat_front_addr":"10.20.193.150:6853/317686896","state":["exists","up"]},{"osd":4,"uuid":"e19f379d-43d8-4536-ac93-ad0d30c98828","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.150:6802","nonce":1337979806},{"type":"v1","addr":"10.20.193.150:6803","nonce":1337979806}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.150:6804","nonce":1337979806},{"type":"v1","addr":"10.20.193.150:6805","nonce":1337979806}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.150:6808","nonce":1337979806},{"type":"v1","addr":"10.20.193.150:6809","nonce":1337979806}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.150:6806","nonce":1337979806},{"type":"v1","addr":"10.20.193.150:6807","nonce":1337979806}]},"public_addr":"10.20.193.150:6803/1337979806","cluster_addr":"10.20.193.150:6805/1337979806","heartbeat_back_addr":"10.20.193.150:6809/1337979806","heartbeat_front_addr":"10.20.193.150:6807/1337979806","state":["exists","up"]},{"osd":5,"uuid":"5d7c9240-c8e9-40d3-835b-ded2969b7c24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.150:6874","nonce":1455759585},{"type":"v1","addr":"10.20.193.150:6875","nonce":1455759585}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.150:6876","nonce":1455759585},{"type":"v1","addr":"10.20.193.150:6877","nonce":1455759585}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.150:6880","nonce":1455759585},{"type":"v1","addr":"10.20.193.150:6881","nonce":1455759585}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.150:6878","nonce":1455759585},{"type":"v1","addr":"10.20.193.150:6879","nonce":1455759585}]},"public_addr":"10.20.193.150:6875/1455759585","cluster_addr":"10.20.193.150:6877/1455759585","heartbeat_back_addr":"10.20.193.150:6881/1455759585","heartbeat_front_addr":"10.20.193.150:6879/1455759585","state":["exists","up"]},{"osd":6,"uuid":"0978d144-b749-4b5e-9c84-9a6c2f1bc3dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.150:6818","nonce":1341103058},{"type":"v1","addr":"10.20.193.150:6819","nonce":1341103058}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.150:6820","nonce":1341103058},{"type":"v1","addr":"10.20.193.150:6821","nonce":1341103058}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.150:6824","nonce":1341103058},{"type":"v1","addr":"10.20.193.150:6825","nonce":1341103058}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.150:6822","nonce":1341103058},{"type":"v1","addr":"10.20.193.150:6823","nonce":1341103058}]},"public_addr":"10.20.193.150:6819/1341103058","cluster_addr":"10.20.193.150:6821/1341103058","heartbeat_back_addr":"10.20.193.150:6825/1341103058","heartbeat_front_addr":"10.20.193.150:6823/1341103058","state":["exists","up"]},{"osd":7,"uuid":"06c1e800-e425-4898-9bbf-fbf1b196d33e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.150:6827","nonce":1453052406},{"type":"v1","addr":"10.20.193.150:6829","nonce":1453052406}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.150:6831","nonce":1453052406},{"type":"v1","addr":"10.20.193.150:6835","nonce":1453052406}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.150:6845","nonce":1453052406},{"type":"v1","addr":"10.20.193.150:6849","nonce":1453052406}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.150:6838","nonce":1453052406},{"type":"v1","addr":"10.20.193.150:6842","nonce":1453052406}]},"public_addr":"10.20.193.150:6829/1453052406","cluster_addr":"10.20.193.150:6835/1453052406","heartbeat_back_addr":"10.20.193.150:6849/1453052406","heartbeat_front_addr":"10.20.193.150:6842/1453052406","state":["exists","up"]},{"osd":8,"uuid":"52f40cc0-0808-4b7d-a711-e77d7d19ff58","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.150:6859","nonce":2701237949},{"type":"v1","addr":"10.20.193.150:6861","nonce":2701237949}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.150:6863","nonce":2701237949},{"type":"v1","addr":"10.20.193.150:6865","nonce":2701237949}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.150:6871","nonce":2701237949},{"type":"v1","addr":"10.20.193.150:6873","nonce":2701237949}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.150:6867","nonce":2701237949},{"type":"v1","addr":"10.20.193.150:6869","nonce":2701237949}]},"public_addr":"10.20.193.150:6861/2701237949","cluster_addr":"10.20.193.150:6865/2701237949","heartbeat_back_addr":"10.20.193.150:6873/2701237949","heartbeat_front_addr":"10.20.193.150:6869/2701237949","state":["exists","up"]},{"osd":9,"uuid":"74e2070c-75ed-4e1b-b25a-86614f45a1fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.150:6832","nonce":2439465072},{"type":"v1","addr":"10.20.193.150:6836","nonce":2439465072}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.150:6841","nonce":2439465072},{"type":"v1","addr":"10.20.193.150:6846","nonce":2439465072}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.150:6854","nonce":2439465072},{"type":"v1","addr":"10.20.193.150:6856","nonce":2439465072}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.150:6850","nonce":2439465072},{"type":"v1","addr":"10.20.193.150:6852","nonce":2439465072}]},"public_addr":"10.20.193.150:6836/2439465072","cluster_addr":"10.20.193.150:6846/2439465072","heartbeat_back_addr":"10.20.193.150:6856/2439465072","heartbeat_front_addr":"10.20.193.150:6852/2439465072","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-21T23:07:05.499158+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-21T23:07:05.477193+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-21T23:07:05.479491+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-21T23:07:05.482361+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-21T23:07:05.561036+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-21T23:07:05.555880+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-21T23:07:05.537802+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-21T23:07:05.553574+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-21T23:07:05.535732+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-21T23:07:05.536298+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"isa","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":1,"snaps":[{"begin":2,"length":1}]}],"new_removed_snaps":[{"pool":1,"snaps":[{"begin":2,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2026-02-21T23:07:16.545 INFO:tasks.ceph.ceph_manager.ceph:all up! 2026-02-21T23:07:16.546 DEBUG:teuthology.orchestra.run.trial150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2026-02-21T23:07:16.726 INFO:teuthology.orchestra.run.trial150.stdout: 2026-02-21T23:07:16.726 INFO:teuthology.orchestra.run.trial150.stdout:{"epoch":20,"fsid":"f77161f3-2466-452f-b0a1-014035b869f9","created":"2026-02-21T23:07:01.892470+0000","modified":"2026-02-21T23:07:15.936364+0000","last_up_change":"2026-02-21T23:07:06.902989+0000","last_in_change":"2026-02-21T23:07:03.803679+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":10,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"tentacle","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2026-02-21T23:07:12.576318+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":20,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"nonprimary_shards":"{}","options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":0.89999997615814209,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1.0000001192092896,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c4ec8e56-7376-4e6a-84af-d03afaa3dbee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.150:6858","nonce":1860146791},{"type":"v1","addr":"10.20.193.150:6860","nonce":1860146791}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.150:6862","nonce":1860146791},{"type":"v1","addr":"10.20.193.150:6864","nonce":1860146791}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.150:6870","nonce":1860146791},{"type":"v1","addr":"10.20.193.150:6872","nonce":1860146791}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.150:6866","nonce":1860146791},{"type":"v1","addr":"10.20.193.150:6868","nonce":1860146791}]},"public_addr":"10.20.193.150:6860/1860146791","cluster_addr":"10.20.193.150:6864/1860146791","heartbeat_back_addr":"10.20.193.150:6872/1860146791","heartbeat_front_addr":"10.20.193.150:6868/1860146791","state":["exists","up"]},{"osd":1,"uuid":"389ddd9e-6beb-48ce-8ee1-574db38a866c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.150:6810","nonce":2256230450},{"type":"v1","addr":"10.20.193.150:6811","nonce":2256230450}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.150:6812","nonce":2256230450},{"type":"v1","addr":"10.20.193.150:6813","nonce":2256230450}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.150:6816","nonce":2256230450},{"type":"v1","addr":"10.20.193.150:6817","nonce":2256230450}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.150:6814","nonce":2256230450},{"type":"v1","addr":"10.20.193.150:6815","nonce":2256230450}]},"public_addr":"10.20.193.150:6811/2256230450","cluster_addr":"10.20.193.150:6813/2256230450","heartbeat_back_addr":"10.20.193.150:6817/2256230450","heartbeat_front_addr":"10.20.193.150:6815/2256230450","state":["exists","up"]},{"osd":2,"uuid":"718e49c5-1294-46b1-b1b4-fa3edda8564c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.150:6826","nonce":3930449501},{"type":"v1","addr":"10.20.193.150:6828","nonce":3930449501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.150:6830","nonce":3930449501},{"type":"v1","addr":"10.20.193.150:6833","nonce":3930449501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.150:6844","nonce":3930449501},{"type":"v1","addr":"10.20.193.150:6847","nonce":3930449501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.150:6837","nonce":3930449501},{"type":"v1","addr":"10.20.193.150:6840","nonce":3930449501}]},"public_addr":"10.20.193.150:6828/3930449501","cluster_addr":"10.20.193.150:6833/3930449501","heartbeat_back_addr":"10.20.193.150:6847/3930449501","heartbeat_front_addr":"10.20.193.150:6840/3930449501","state":["exists","up"]},{"osd":3,"uuid":"9a09d418-342f-4db0-a6e1-700059b6f0f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.150:6834","nonce":317686896},{"type":"v1","addr":"10.20.193.150:6839","nonce":317686896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.150:6843","nonce":317686896},{"type":"v1","addr":"10.20.193.150:6848","nonce":317686896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.150:6855","nonce":317686896},{"type":"v1","addr":"10.20.193.150:6857","nonce":317686896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.150:6851","nonce":317686896},{"type":"v1","addr":"10.20.193.150:6853","nonce":317686896}]},"public_addr":"10.20.193.150:6839/317686896","cluster_addr":"10.20.193.150:6848/317686896","heartbeat_back_addr":"10.20.193.150:6857/317686896","heartbeat_front_addr":"10.20.193.150:6853/317686896","state":["exists","up"]},{"osd":4,"uuid":"e19f379d-43d8-4536-ac93-ad0d30c98828","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.150:6802","nonce":1337979806},{"type":"v1","addr":"10.20.193.150:6803","nonce":1337979806}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.150:6804","nonce":1337979806},{"type":"v1","addr":"10.20.193.150:6805","nonce":1337979806}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.150:6808","nonce":1337979806},{"type":"v1","addr":"10.20.193.150:6809","nonce":1337979806}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.150:6806","nonce":1337979806},{"type":"v1","addr":"10.20.193.150:6807","nonce":1337979806}]},"public_addr":"10.20.193.150:6803/1337979806","cluster_addr":"10.20.193.150:6805/1337979806","heartbeat_back_addr":"10.20.193.150:6809/1337979806","heartbeat_front_addr":"10.20.193.150:6807/1337979806","state":["exists","up"]},{"osd":5,"uuid":"5d7c9240-c8e9-40d3-835b-ded2969b7c24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.150:6874","nonce":1455759585},{"type":"v1","addr":"10.20.193.150:6875","nonce":1455759585}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.150:6876","nonce":1455759585},{"type":"v1","addr":"10.20.193.150:6877","nonce":1455759585}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.150:6880","nonce":1455759585},{"type":"v1","addr":"10.20.193.150:6881","nonce":1455759585}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.150:6878","nonce":1455759585},{"type":"v1","addr":"10.20.193.150:6879","nonce":1455759585}]},"public_addr":"10.20.193.150:6875/1455759585","cluster_addr":"10.20.193.150:6877/1455759585","heartbeat_back_addr":"10.20.193.150:6881/1455759585","heartbeat_front_addr":"10.20.193.150:6879/1455759585","state":["exists","up"]},{"osd":6,"uuid":"0978d144-b749-4b5e-9c84-9a6c2f1bc3dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.150:6818","nonce":1341103058},{"type":"v1","addr":"10.20.193.150:6819","nonce":1341103058}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.150:6820","nonce":1341103058},{"type":"v1","addr":"10.20.193.150:6821","nonce":1341103058}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.150:6824","nonce":1341103058},{"type":"v1","addr":"10.20.193.150:6825","nonce":1341103058}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.150:6822","nonce":1341103058},{"type":"v1","addr":"10.20.193.150:6823","nonce":1341103058}]},"public_addr":"10.20.193.150:6819/1341103058","cluster_addr":"10.20.193.150:6821/1341103058","heartbeat_back_addr":"10.20.193.150:6825/1341103058","heartbeat_front_addr":"10.20.193.150:6823/1341103058","state":["exists","up"]},{"osd":7,"uuid":"06c1e800-e425-4898-9bbf-fbf1b196d33e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.150:6827","nonce":1453052406},{"type":"v1","addr":"10.20.193.150:6829","nonce":1453052406}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.150:6831","nonce":1453052406},{"type":"v1","addr":"10.20.193.150:6835","nonce":1453052406}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.150:6845","nonce":1453052406},{"type":"v1","addr":"10.20.193.150:6849","nonce":1453052406}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.150:6838","nonce":1453052406},{"type":"v1","addr":"10.20.193.150:6842","nonce":1453052406}]},"public_addr":"10.20.193.150:6829/1453052406","cluster_addr":"10.20.193.150:6835/1453052406","heartbeat_back_addr":"10.20.193.150:6849/1453052406","heartbeat_front_addr":"10.20.193.150:6842/1453052406","state":["exists","up"]},{"osd":8,"uuid":"52f40cc0-0808-4b7d-a711-e77d7d19ff58","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.150:6859","nonce":2701237949},{"type":"v1","addr":"10.20.193.150:6861","nonce":2701237949}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.150:6863","nonce":2701237949},{"type":"v1","addr":"10.20.193.150:6865","nonce":2701237949}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.150:6871","nonce":2701237949},{"type":"v1","addr":"10.20.193.150:6873","nonce":2701237949}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.150:6867","nonce":2701237949},{"type":"v1","addr":"10.20.193.150:6869","nonce":2701237949}]},"public_addr":"10.20.193.150:6861/2701237949","cluster_addr":"10.20.193.150:6865/2701237949","heartbeat_back_addr":"10.20.193.150:6873/2701237949","heartbeat_front_addr":"10.20.193.150:6869/2701237949","state":["exists","up"]},{"osd":9,"uuid":"74e2070c-75ed-4e1b-b25a-86614f45a1fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.150:6832","nonce":2439465072},{"type":"v1","addr":"10.20.193.150:6836","nonce":2439465072}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.150:6841","nonce":2439465072},{"type":"v1","addr":"10.20.193.150:6846","nonce":2439465072}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.150:6854","nonce":2439465072},{"type":"v1","addr":"10.20.193.150:6856","nonce":2439465072}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.150:6850","nonce":2439465072},{"type":"v1","addr":"10.20.193.150:6852","nonce":2439465072}]},"public_addr":"10.20.193.150:6836/2439465072","cluster_addr":"10.20.193.150:6846/2439465072","heartbeat_back_addr":"10.20.193.150:6856/2439465072","heartbeat_front_addr":"10.20.193.150:6852/2439465072","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-21T23:07:05.499158+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-21T23:07:05.477193+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-21T23:07:05.479491+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-21T23:07:05.482361+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-21T23:07:05.561036+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-21T23:07:05.555880+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-21T23:07:05.537802+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-21T23:07:05.553574+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-21T23:07:05.535732+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-21T23:07:05.536298+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"isa","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":1,"snaps":[{"begin":2,"length":1}]}],"new_removed_snaps":[{"pool":1,"snaps":[{"begin":2,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2026-02-21T23:07:16.736 DEBUG:teuthology.orchestra.run.trial150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2026-02-21T23:07:16.736 DEBUG:teuthology.orchestra.run.trial150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2026-02-21T23:07:16.737 DEBUG:teuthology.orchestra.run.trial150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2026-02-21T23:07:16.737 DEBUG:teuthology.orchestra.run.trial150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2026-02-21T23:07:16.738 DEBUG:teuthology.orchestra.run.trial150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2026-02-21T23:07:16.738 DEBUG:teuthology.orchestra.run.trial150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2026-02-21T23:07:16.739 DEBUG:teuthology.orchestra.run.trial150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2026-02-21T23:07:16.739 DEBUG:teuthology.orchestra.run.trial150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2026-02-21T23:07:16.740 DEBUG:teuthology.orchestra.run.trial150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.8 flush_pg_stats 2026-02-21T23:07:16.740 DEBUG:teuthology.orchestra.run.trial150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.9 flush_pg_stats 2026-02-21T23:07:16.838 INFO:teuthology.orchestra.run.trial150.stdout:64424509444 2026-02-21T23:07:16.839 DEBUG:teuthology.orchestra.run.trial150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2026-02-21T23:07:16.840 INFO:teuthology.orchestra.run.trial150.stdout:64424509443 2026-02-21T23:07:16.840 DEBUG:teuthology.orchestra.run.trial150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2026-02-21T23:07:16.842 INFO:teuthology.orchestra.run.trial150.stdout:64424509444 2026-02-21T23:07:16.843 DEBUG:teuthology.orchestra.run.trial150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2026-02-21T23:07:16.844 INFO:teuthology.orchestra.run.trial150.stdout:64424509444 2026-02-21T23:07:16.844 DEBUG:teuthology.orchestra.run.trial150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2026-02-21T23:07:16.845 INFO:teuthology.orchestra.run.trial150.stdout:64424509443 2026-02-21T23:07:16.846 DEBUG:teuthology.orchestra.run.trial150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2026-02-21T23:07:16.847 INFO:teuthology.orchestra.run.trial150.stdout:64424509444 2026-02-21T23:07:16.848 DEBUG:teuthology.orchestra.run.trial150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2026-02-21T23:07:16.849 INFO:teuthology.orchestra.run.trial150.stdout:64424509443 2026-02-21T23:07:16.850 INFO:teuthology.orchestra.run.trial150.stdout:64424509444 2026-02-21T23:07:16.850 DEBUG:teuthology.orchestra.run.trial150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2026-02-21T23:07:16.851 DEBUG:teuthology.orchestra.run.trial150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.8 2026-02-21T23:07:16.852 INFO:teuthology.orchestra.run.trial150.stdout:64424509444 2026-02-21T23:07:16.853 DEBUG:teuthology.orchestra.run.trial150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2026-02-21T23:07:16.854 INFO:teuthology.orchestra.run.trial150.stdout:64424509443 2026-02-21T23:07:16.854 DEBUG:teuthology.orchestra.run.trial150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.9 2026-02-21T23:07:17.038 INFO:teuthology.orchestra.run.trial150.stdout:64424509443 2026-02-21T23:07:17.039 INFO:teuthology.orchestra.run.trial150.stdout:64424509442 2026-02-21T23:07:17.050 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509444 got 64424509443 for osd.0 2026-02-21T23:07:17.051 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509443 got 64424509442 for osd.4 2026-02-21T23:07:17.080 INFO:teuthology.orchestra.run.trial150.stdout:64424509442 2026-02-21T23:07:17.084 INFO:teuthology.orchestra.run.trial150.stdout:64424509443 2026-02-21T23:07:17.087 INFO:teuthology.orchestra.run.trial150.stdout:64424509443 2026-02-21T23:07:17.091 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509443 got 64424509442 for osd.3 2026-02-21T23:07:17.095 INFO:teuthology.orchestra.run.trial150.stdout:64424509443 2026-02-21T23:07:17.096 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509444 got 64424509443 for osd.1 2026-02-21T23:07:17.097 INFO:teuthology.orchestra.run.trial150.stdout:64424509443 2026-02-21T23:07:17.099 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509444 got 64424509443 for osd.2 2026-02-21T23:07:17.099 INFO:teuthology.orchestra.run.trial150.stdout:64424509442 2026-02-21T23:07:17.103 INFO:teuthology.orchestra.run.trial150.stdout:64424509442 2026-02-21T23:07:17.103 INFO:teuthology.orchestra.run.trial150.stdout:64424509443 2026-02-21T23:07:17.107 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509444 got 64424509443 for osd.7 2026-02-21T23:07:17.109 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509444 got 64424509443 for osd.5 2026-02-21T23:07:17.110 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509443 got 64424509442 for osd.6 2026-02-21T23:07:17.113 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509443 got 64424509442 for osd.9 2026-02-21T23:07:17.114 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509444 got 64424509443 for osd.8 2026-02-21T23:07:18.051 DEBUG:teuthology.orchestra.run.trial150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2026-02-21T23:07:18.052 DEBUG:teuthology.orchestra.run.trial150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2026-02-21T23:07:18.091 DEBUG:teuthology.orchestra.run.trial150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2026-02-21T23:07:18.097 DEBUG:teuthology.orchestra.run.trial150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2026-02-21T23:07:18.099 DEBUG:teuthology.orchestra.run.trial150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2026-02-21T23:07:18.107 DEBUG:teuthology.orchestra.run.trial150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2026-02-21T23:07:18.110 DEBUG:teuthology.orchestra.run.trial150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2026-02-21T23:07:18.110 DEBUG:teuthology.orchestra.run.trial150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2026-02-21T23:07:18.114 DEBUG:teuthology.orchestra.run.trial150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.9 2026-02-21T23:07:18.115 DEBUG:teuthology.orchestra.run.trial150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.8 2026-02-21T23:07:18.248 INFO:teuthology.orchestra.run.trial150.stdout:64424509442 2026-02-21T23:07:18.251 INFO:teuthology.orchestra.run.trial150.stdout:64424509443 2026-02-21T23:07:18.259 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509443 got 64424509442 for osd.4 2026-02-21T23:07:18.262 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509444 got 64424509443 for osd.0 2026-02-21T23:07:18.295 INFO:teuthology.orchestra.run.trial150.stdout:64424509443 2026-02-21T23:07:18.296 INFO:teuthology.orchestra.run.trial150.stdout:64424509443 2026-02-21T23:07:18.299 INFO:teuthology.orchestra.run.trial150.stdout:64424509442 2026-02-21T23:07:18.307 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509444 got 64424509443 for osd.1 2026-02-21T23:07:18.308 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509444 got 64424509443 for osd.2 2026-02-21T23:07:18.310 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509443 got 64424509442 for osd.3 2026-02-21T23:07:18.339 INFO:teuthology.orchestra.run.trial150.stdout:64424509442 2026-02-21T23:07:18.341 INFO:teuthology.orchestra.run.trial150.stdout:64424509443 2026-02-21T23:07:18.343 INFO:teuthology.orchestra.run.trial150.stdout:64424509443 2026-02-21T23:07:18.344 INFO:teuthology.orchestra.run.trial150.stdout:64424509443 2026-02-21T23:07:18.349 INFO:teuthology.orchestra.run.trial150.stdout:64424509442 2026-02-21T23:07:18.351 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509443 got 64424509442 for osd.6 2026-02-21T23:07:18.352 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509444 got 64424509443 for osd.5 2026-02-21T23:07:18.356 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509444 got 64424509443 for osd.7 2026-02-21T23:07:18.357 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509444 got 64424509443 for osd.8 2026-02-21T23:07:18.359 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509443 got 64424509442 for osd.9 2026-02-21T23:07:19.260 DEBUG:teuthology.orchestra.run.trial150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2026-02-21T23:07:19.263 DEBUG:teuthology.orchestra.run.trial150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2026-02-21T23:07:19.309 DEBUG:teuthology.orchestra.run.trial150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2026-02-21T23:07:19.309 DEBUG:teuthology.orchestra.run.trial150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2026-02-21T23:07:19.311 DEBUG:teuthology.orchestra.run.trial150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2026-02-21T23:07:19.352 DEBUG:teuthology.orchestra.run.trial150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2026-02-21T23:07:19.353 DEBUG:teuthology.orchestra.run.trial150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2026-02-21T23:07:19.357 DEBUG:teuthology.orchestra.run.trial150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2026-02-21T23:07:19.357 DEBUG:teuthology.orchestra.run.trial150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.8 2026-02-21T23:07:19.360 DEBUG:teuthology.orchestra.run.trial150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.9 2026-02-21T23:07:19.449 INFO:teuthology.orchestra.run.trial150.stdout:64424509444 2026-02-21T23:07:19.458 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509443 got 64424509444 for osd.4 2026-02-21T23:07:19.458 DEBUG:teuthology.parallel:result is None 2026-02-21T23:07:19.489 INFO:teuthology.orchestra.run.trial150.stdout:64424509445 2026-02-21T23:07:19.501 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509444 got 64424509445 for osd.0 2026-02-21T23:07:19.501 DEBUG:teuthology.parallel:result is None 2026-02-21T23:07:19.538 INFO:teuthology.orchestra.run.trial150.stdout:64424509444 2026-02-21T23:07:19.540 INFO:teuthology.orchestra.run.trial150.stdout:64424509445 2026-02-21T23:07:19.542 INFO:teuthology.orchestra.run.trial150.stdout:64424509445 2026-02-21T23:07:19.550 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509443 got 64424509444 for osd.3 2026-02-21T23:07:19.550 DEBUG:teuthology.parallel:result is None 2026-02-21T23:07:19.553 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509444 got 64424509445 for osd.1 2026-02-21T23:07:19.554 DEBUG:teuthology.parallel:result is None 2026-02-21T23:07:19.555 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509444 got 64424509445 for osd.2 2026-02-21T23:07:19.555 DEBUG:teuthology.parallel:result is None 2026-02-21T23:07:19.577 INFO:teuthology.orchestra.run.trial150.stdout:64424509445 2026-02-21T23:07:19.579 INFO:teuthology.orchestra.run.trial150.stdout:64424509445 2026-02-21T23:07:19.583 INFO:teuthology.orchestra.run.trial150.stdout:64424509444 2026-02-21T23:07:19.585 INFO:teuthology.orchestra.run.trial150.stdout:64424509444 2026-02-21T23:07:19.588 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509444 got 64424509445 for osd.8 2026-02-21T23:07:19.589 DEBUG:teuthology.parallel:result is None 2026-02-21T23:07:19.591 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509444 got 64424509445 for osd.7 2026-02-21T23:07:19.591 INFO:teuthology.orchestra.run.trial150.stdout:64424509445 2026-02-21T23:07:19.591 DEBUG:teuthology.parallel:result is None 2026-02-21T23:07:19.594 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509443 got 64424509444 for osd.6 2026-02-21T23:07:19.594 DEBUG:teuthology.parallel:result is None 2026-02-21T23:07:19.596 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509443 got 64424509444 for osd.9 2026-02-21T23:07:19.596 DEBUG:teuthology.parallel:result is None 2026-02-21T23:07:19.600 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509444 got 64424509445 for osd.5 2026-02-21T23:07:19.601 DEBUG:teuthology.parallel:result is None 2026-02-21T23:07:19.601 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2026-02-21T23:07:19.601 DEBUG:teuthology.orchestra.run.trial150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T23:07:19.815 INFO:teuthology.orchestra.run.trial150.stdout: 2026-02-21T23:07:19.816 INFO:teuthology.orchestra.run.trial150.stderr:dumped all 2026-02-21T23:07:19.823 INFO:teuthology.orchestra.run.trial150.stdout:{"pg_ready":true,"pg_map":{"version":26,"stamp":"2026-02-21T23:07:18.815232+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":14,"num_osds":10,"num_per_pool_osds":10,"num_per_pool_omap_osds":9,"kb":1048576000,"kb_used":271528,"kb_used_data":1928,"kb_used_omap":72,"kb_used_meta":269367,"kb_avail":1048304472,"statfs":{"total":1073741824000,"available":1073463779328,"internally_reserved":0,"allocated":1974272,"data_stored":655858,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":74096,"internal_metadata":275832464},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":20,"reported_epoch":20,"state":"active+clean","last_fresh":"2026-02-21T23:07:16.241338+0000","last_change":"2026-02-21T23:07:16.241338+0000","last_active":"2026-02-21T23:07:16.241338+0000","last_peered":"2026-02-21T23:07:16.241338+0000","last_clean":"2026-02-21T23:07:16.241338+0000","last_became_active":"2026-02-21T23:07:13.957901+0000","last_became_peered":"2026-02-21T23:07:13.957901+0000","last_unstale":"2026-02-21T23:07:16.241338+0000","last_undegraded":"2026-02-21T23:07:16.241338+0000","last_fullsized":"2026-02-21T23:07:16.241338+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T23:07:12.930820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T23:07:12.930820+0000","last_clean_scrub_stamp":"2026-02-21T23:07:12.930820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T10:01:00.850150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00033767299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.6","version":"0'0","reported_seq":20,"reported_epoch":20,"state":"active+clean","last_fresh":"2026-02-21T23:07:16.241210+0000","last_change":"2026-02-21T23:07:16.241210+0000","last_active":"2026-02-21T23:07:16.241210+0000","last_peered":"2026-02-21T23:07:16.241210+0000","last_clean":"2026-02-21T23:07:16.241210+0000","last_became_active":"2026-02-21T23:07:13.958141+0000","last_became_peered":"2026-02-21T23:07:13.958141+0000","last_unstale":"2026-02-21T23:07:16.241210+0000","last_undegraded":"2026-02-21T23:07:16.241210+0000","last_fullsized":"2026-02-21T23:07:16.241210+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T23:07:12.930820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T23:07:12.930820+0000","last_clean_scrub_stamp":"2026-02-21T23:07:12.930820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T02:40:52.081250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022801700000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9],"acting":[6,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.5","version":"0'0","reported_seq":20,"reported_epoch":20,"state":"active+clean","last_fresh":"2026-02-21T23:07:16.241298+0000","last_change":"2026-02-21T23:07:16.241298+0000","last_active":"2026-02-21T23:07:16.241298+0000","last_peered":"2026-02-21T23:07:16.241298+0000","last_clean":"2026-02-21T23:07:16.241298+0000","last_became_active":"2026-02-21T23:07:13.935947+0000","last_became_peered":"2026-02-21T23:07:13.935947+0000","last_unstale":"2026-02-21T23:07:16.241298+0000","last_undegraded":"2026-02-21T23:07:16.241298+0000","last_fullsized":"2026-02-21T23:07:16.241298+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T23:07:12.930820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T23:07:12.930820+0000","last_clean_scrub_stamp":"2026-02-21T23:07:12.930820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T07:49:56.767234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00029313000000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8],"acting":[4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.4","version":"0'0","reported_seq":20,"reported_epoch":20,"state":"active+clean","last_fresh":"2026-02-21T23:07:16.241047+0000","last_change":"2026-02-21T23:07:16.241047+0000","last_active":"2026-02-21T23:07:16.241047+0000","last_peered":"2026-02-21T23:07:16.241047+0000","last_clean":"2026-02-21T23:07:16.241047+0000","last_became_active":"2026-02-21T23:07:13.935926+0000","last_became_peered":"2026-02-21T23:07:13.935926+0000","last_unstale":"2026-02-21T23:07:16.241047+0000","last_undegraded":"2026-02-21T23:07:16.241047+0000","last_fullsized":"2026-02-21T23:07:16.241047+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T23:07:12.930820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T23:07:12.930820+0000","last_clean_scrub_stamp":"2026-02-21T23:07:12.930820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:56:50.306005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025179199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.3","version":"18'1","reported_seq":21,"reported_epoch":20,"state":"active+clean","last_fresh":"2026-02-21T23:07:16.241303+0000","last_change":"2026-02-21T23:07:16.241303+0000","last_active":"2026-02-21T23:07:16.241303+0000","last_peered":"2026-02-21T23:07:16.241303+0000","last_clean":"2026-02-21T23:07:16.241303+0000","last_became_active":"2026-02-21T23:07:13.958128+0000","last_became_peered":"2026-02-21T23:07:13.958128+0000","last_unstale":"2026-02-21T23:07:16.241303+0000","last_undegraded":"2026-02-21T23:07:16.241303+0000","last_fullsized":"2026-02-21T23:07:16.241303+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T23:07:12.930820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T23:07:12.930820+0000","last_clean_scrub_stamp":"2026-02-21T23:07:12.930820+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T07:12:01.474062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00032599000000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.2","version":"20'2","reported_seq":22,"reported_epoch":20,"state":"active+clean","last_fresh":"2026-02-21T23:07:15.940055+0000","last_change":"2026-02-21T23:07:15.939669+0000","last_active":"2026-02-21T23:07:15.940055+0000","last_peered":"2026-02-21T23:07:15.940055+0000","last_clean":"2026-02-21T23:07:15.940055+0000","last_became_active":"2026-02-21T23:07:13.957905+0000","last_became_peered":"2026-02-21T23:07:13.957905+0000","last_unstale":"2026-02-21T23:07:15.940055+0000","last_undegraded":"2026-02-21T23:07:15.940055+0000","last_fullsized":"2026-02-21T23:07:15.940055+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T23:07:12.930820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T23:07:12.930820+0000","last_clean_scrub_stamp":"2026-02-21T23:07:12.930820+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T07:29:22.018670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044592600000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.1","version":"0'0","reported_seq":20,"reported_epoch":20,"state":"active+clean","last_fresh":"2026-02-21T23:07:15.938035+0000","last_change":"2026-02-21T23:07:15.938035+0000","last_active":"2026-02-21T23:07:15.938035+0000","last_peered":"2026-02-21T23:07:15.938035+0000","last_clean":"2026-02-21T23:07:15.938035+0000","last_became_active":"2026-02-21T23:07:13.935791+0000","last_became_peered":"2026-02-21T23:07:13.935791+0000","last_unstale":"2026-02-21T23:07:15.938035+0000","last_undegraded":"2026-02-21T23:07:15.938035+0000","last_fullsized":"2026-02-21T23:07:15.938035+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T23:07:12.930820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T23:07:12.930820+0000","last_clean_scrub_stamp":"2026-02-21T23:07:12.930820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T07:45:36.793166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00037064500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8],"acting":[5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"0'0","reported_seq":20,"reported_epoch":20,"state":"active+clean","last_fresh":"2026-02-21T23:07:16.241292+0000","last_change":"2026-02-21T23:07:16.241292+0000","last_active":"2026-02-21T23:07:16.241292+0000","last_peered":"2026-02-21T23:07:16.241292+0000","last_clean":"2026-02-21T23:07:16.241292+0000","last_became_active":"2026-02-21T23:07:13.936270+0000","last_became_peered":"2026-02-21T23:07:13.936270+0000","last_unstale":"2026-02-21T23:07:16.241292+0000","last_undegraded":"2026-02-21T23:07:16.241292+0000","last_fullsized":"2026-02-21T23:07:16.241292+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T23:07:12.930820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T23:07:12.930820+0000","last_clean_scrub_stamp":"2026-02-21T23:07:12.930820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:56:50.306005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023912799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0],"acting":[8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[{"start":"2","length":"1"}]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":9}],"osd_stats":[{"osd":9,"up_from":15,"seq":64424509444,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":27152,"kb_used_data":192,"kb_used_omap":7,"kb_used_meta":26936,"kb_avail":104830448,"statfs":{"total":107374182400,"available":107346378752,"internally_reserved":0,"allocated":196608,"data_stored":65582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":7479,"internal_metadata":27583177},"hb_peers":[0,1,2,3,4,5,6,7,8],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":15,"seq":64424509445,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":27152,"kb_used_data":192,"kb_used_omap":7,"kb_used_meta":26936,"kb_avail":104830448,"statfs":{"total":107374182400,"available":107346378752,"internally_reserved":0,"allocated":196608,"data_stored":65582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":7475,"internal_metadata":27583181},"hb_peers":[0,1,2,3,4,5,6,7,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":15,"seq":64424509445,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":27156,"kb_used_data":196,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":104830444,"statfs":{"total":107374182400,"available":107346374656,"internally_reserved":0,"allocated":200704,"data_stored":65601,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6820,"internal_metadata":27583836},"hb_peers":[0,1,2,3,4,5,6,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509444,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":27152,"kb_used_data":192,"kb_used_omap":7,"kb_used_meta":26936,"kb_avail":104830448,"statfs":{"total":107374182400,"available":107346378752,"internally_reserved":0,"allocated":196608,"data_stored":65582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":7474,"internal_metadata":27583182},"hb_peers":[0,1,2,3,4,5,7,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509445,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":27152,"kb_used_data":192,"kb_used_omap":7,"kb_used_meta":26936,"kb_avail":104830448,"statfs":{"total":107374182400,"available":107346378752,"internally_reserved":0,"allocated":196608,"data_stored":65582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":7475,"internal_metadata":27583181},"hb_peers":[0,1,2,3,4,6,7,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509444,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":27152,"kb_used_data":192,"kb_used_omap":7,"kb_used_meta":26936,"kb_avail":104830448,"statfs":{"total":107374182400,"available":107346378752,"internally_reserved":0,"allocated":196608,"data_stored":65582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":7475,"internal_metadata":27583181},"hb_peers":[0,1,2,3,5,6,7,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509444,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":27152,"kb_used_data":192,"kb_used_omap":7,"kb_used_meta":26936,"kb_avail":104830448,"statfs":{"total":107374182400,"available":107346378752,"internally_reserved":0,"allocated":196608,"data_stored":65582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":7473,"internal_metadata":27583183},"hb_peers":[0,1,2,4,5,6,7,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509445,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":104857600,"kb_used":27152,"kb_used_data":192,"kb_used_omap":7,"kb_used_meta":26936,"kb_avail":104830448,"statfs":{"total":107374182400,"available":107346378752,"internally_reserved":0,"allocated":196608,"data_stored":65582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":7475,"internal_metadata":27583181},"hb_peers":[0,1,3,4,5,6,7,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509445,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":27152,"kb_used_data":192,"kb_used_omap":7,"kb_used_meta":26936,"kb_avail":104830448,"statfs":{"total":107374182400,"available":107346378752,"internally_reserved":0,"allocated":196608,"data_stored":65582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":7475,"internal_metadata":27583181},"hb_peers":[0,2,3,4,5,6,7,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509445,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":27156,"kb_used_data":196,"kb_used_omap":7,"kb_used_meta":26936,"kb_avail":104830444,"statfs":{"total":107374182400,"available":107346374656,"internally_reserved":0,"allocated":200704,"data_stored":65601,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":7475,"internal_metadata":27583181},"hb_peers":[1,2,3,4,5,6,7,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T23:07:19.825 DEBUG:teuthology.orchestra.run.trial150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T23:07:19.998 INFO:teuthology.orchestra.run.trial150.stdout: 2026-02-21T23:07:19.998 INFO:teuthology.orchestra.run.trial150.stderr:dumped all 2026-02-21T23:07:20.007 INFO:teuthology.orchestra.run.trial150.stdout:{"pg_ready":true,"pg_map":{"version":26,"stamp":"2026-02-21T23:07:18.815232+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":14,"num_osds":10,"num_per_pool_osds":10,"num_per_pool_omap_osds":9,"kb":1048576000,"kb_used":271528,"kb_used_data":1928,"kb_used_omap":72,"kb_used_meta":269367,"kb_avail":1048304472,"statfs":{"total":1073741824000,"available":1073463779328,"internally_reserved":0,"allocated":1974272,"data_stored":655858,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":74096,"internal_metadata":275832464},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":20,"reported_epoch":20,"state":"active+clean","last_fresh":"2026-02-21T23:07:16.241338+0000","last_change":"2026-02-21T23:07:16.241338+0000","last_active":"2026-02-21T23:07:16.241338+0000","last_peered":"2026-02-21T23:07:16.241338+0000","last_clean":"2026-02-21T23:07:16.241338+0000","last_became_active":"2026-02-21T23:07:13.957901+0000","last_became_peered":"2026-02-21T23:07:13.957901+0000","last_unstale":"2026-02-21T23:07:16.241338+0000","last_undegraded":"2026-02-21T23:07:16.241338+0000","last_fullsized":"2026-02-21T23:07:16.241338+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T23:07:12.930820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T23:07:12.930820+0000","last_clean_scrub_stamp":"2026-02-21T23:07:12.930820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T10:01:00.850150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00033767299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.6","version":"0'0","reported_seq":20,"reported_epoch":20,"state":"active+clean","last_fresh":"2026-02-21T23:07:16.241210+0000","last_change":"2026-02-21T23:07:16.241210+0000","last_active":"2026-02-21T23:07:16.241210+0000","last_peered":"2026-02-21T23:07:16.241210+0000","last_clean":"2026-02-21T23:07:16.241210+0000","last_became_active":"2026-02-21T23:07:13.958141+0000","last_became_peered":"2026-02-21T23:07:13.958141+0000","last_unstale":"2026-02-21T23:07:16.241210+0000","last_undegraded":"2026-02-21T23:07:16.241210+0000","last_fullsized":"2026-02-21T23:07:16.241210+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T23:07:12.930820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T23:07:12.930820+0000","last_clean_scrub_stamp":"2026-02-21T23:07:12.930820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T02:40:52.081250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022801700000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9],"acting":[6,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.5","version":"0'0","reported_seq":20,"reported_epoch":20,"state":"active+clean","last_fresh":"2026-02-21T23:07:16.241298+0000","last_change":"2026-02-21T23:07:16.241298+0000","last_active":"2026-02-21T23:07:16.241298+0000","last_peered":"2026-02-21T23:07:16.241298+0000","last_clean":"2026-02-21T23:07:16.241298+0000","last_became_active":"2026-02-21T23:07:13.935947+0000","last_became_peered":"2026-02-21T23:07:13.935947+0000","last_unstale":"2026-02-21T23:07:16.241298+0000","last_undegraded":"2026-02-21T23:07:16.241298+0000","last_fullsized":"2026-02-21T23:07:16.241298+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T23:07:12.930820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T23:07:12.930820+0000","last_clean_scrub_stamp":"2026-02-21T23:07:12.930820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T07:49:56.767234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00029313000000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8],"acting":[4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.4","version":"0'0","reported_seq":20,"reported_epoch":20,"state":"active+clean","last_fresh":"2026-02-21T23:07:16.241047+0000","last_change":"2026-02-21T23:07:16.241047+0000","last_active":"2026-02-21T23:07:16.241047+0000","last_peered":"2026-02-21T23:07:16.241047+0000","last_clean":"2026-02-21T23:07:16.241047+0000","last_became_active":"2026-02-21T23:07:13.935926+0000","last_became_peered":"2026-02-21T23:07:13.935926+0000","last_unstale":"2026-02-21T23:07:16.241047+0000","last_undegraded":"2026-02-21T23:07:16.241047+0000","last_fullsized":"2026-02-21T23:07:16.241047+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T23:07:12.930820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T23:07:12.930820+0000","last_clean_scrub_stamp":"2026-02-21T23:07:12.930820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:56:50.306005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025179199999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.3","version":"18'1","reported_seq":21,"reported_epoch":20,"state":"active+clean","last_fresh":"2026-02-21T23:07:16.241303+0000","last_change":"2026-02-21T23:07:16.241303+0000","last_active":"2026-02-21T23:07:16.241303+0000","last_peered":"2026-02-21T23:07:16.241303+0000","last_clean":"2026-02-21T23:07:16.241303+0000","last_became_active":"2026-02-21T23:07:13.958128+0000","last_became_peered":"2026-02-21T23:07:13.958128+0000","last_unstale":"2026-02-21T23:07:16.241303+0000","last_undegraded":"2026-02-21T23:07:16.241303+0000","last_fullsized":"2026-02-21T23:07:16.241303+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T23:07:12.930820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T23:07:12.930820+0000","last_clean_scrub_stamp":"2026-02-21T23:07:12.930820+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T07:12:01.474062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00032599000000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.2","version":"20'2","reported_seq":22,"reported_epoch":20,"state":"active+clean","last_fresh":"2026-02-21T23:07:15.940055+0000","last_change":"2026-02-21T23:07:15.939669+0000","last_active":"2026-02-21T23:07:15.940055+0000","last_peered":"2026-02-21T23:07:15.940055+0000","last_clean":"2026-02-21T23:07:15.940055+0000","last_became_active":"2026-02-21T23:07:13.957905+0000","last_became_peered":"2026-02-21T23:07:13.957905+0000","last_unstale":"2026-02-21T23:07:15.940055+0000","last_undegraded":"2026-02-21T23:07:15.940055+0000","last_fullsized":"2026-02-21T23:07:15.940055+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T23:07:12.930820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T23:07:12.930820+0000","last_clean_scrub_stamp":"2026-02-21T23:07:12.930820+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T07:29:22.018670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044592600000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.1","version":"0'0","reported_seq":20,"reported_epoch":20,"state":"active+clean","last_fresh":"2026-02-21T23:07:15.938035+0000","last_change":"2026-02-21T23:07:15.938035+0000","last_active":"2026-02-21T23:07:15.938035+0000","last_peered":"2026-02-21T23:07:15.938035+0000","last_clean":"2026-02-21T23:07:15.938035+0000","last_became_active":"2026-02-21T23:07:13.935791+0000","last_became_peered":"2026-02-21T23:07:13.935791+0000","last_unstale":"2026-02-21T23:07:15.938035+0000","last_undegraded":"2026-02-21T23:07:15.938035+0000","last_fullsized":"2026-02-21T23:07:15.938035+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T23:07:12.930820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T23:07:12.930820+0000","last_clean_scrub_stamp":"2026-02-21T23:07:12.930820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T07:45:36.793166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00037064500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8],"acting":[5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"0'0","reported_seq":20,"reported_epoch":20,"state":"active+clean","last_fresh":"2026-02-21T23:07:16.241292+0000","last_change":"2026-02-21T23:07:16.241292+0000","last_active":"2026-02-21T23:07:16.241292+0000","last_peered":"2026-02-21T23:07:16.241292+0000","last_clean":"2026-02-21T23:07:16.241292+0000","last_became_active":"2026-02-21T23:07:13.936270+0000","last_became_peered":"2026-02-21T23:07:13.936270+0000","last_unstale":"2026-02-21T23:07:16.241292+0000","last_undegraded":"2026-02-21T23:07:16.241292+0000","last_fullsized":"2026-02-21T23:07:16.241292+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T23:07:12.930820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T23:07:12.930820+0000","last_clean_scrub_stamp":"2026-02-21T23:07:12.930820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:56:50.306005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00023912799999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0],"acting":[8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[{"start":"2","length":"1"}]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":9}],"osd_stats":[{"osd":9,"up_from":15,"seq":64424509444,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":27152,"kb_used_data":192,"kb_used_omap":7,"kb_used_meta":26936,"kb_avail":104830448,"statfs":{"total":107374182400,"available":107346378752,"internally_reserved":0,"allocated":196608,"data_stored":65582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":7479,"internal_metadata":27583177},"hb_peers":[0,1,2,3,4,5,6,7,8],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":15,"seq":64424509445,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":27152,"kb_used_data":192,"kb_used_omap":7,"kb_used_meta":26936,"kb_avail":104830448,"statfs":{"total":107374182400,"available":107346378752,"internally_reserved":0,"allocated":196608,"data_stored":65582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":7475,"internal_metadata":27583181},"hb_peers":[0,1,2,3,4,5,6,7,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":15,"seq":64424509445,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":27156,"kb_used_data":196,"kb_used_omap":6,"kb_used_meta":26937,"kb_avail":104830444,"statfs":{"total":107374182400,"available":107346374656,"internally_reserved":0,"allocated":200704,"data_stored":65601,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6820,"internal_metadata":27583836},"hb_peers":[0,1,2,3,4,5,6,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509444,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":27152,"kb_used_data":192,"kb_used_omap":7,"kb_used_meta":26936,"kb_avail":104830448,"statfs":{"total":107374182400,"available":107346378752,"internally_reserved":0,"allocated":196608,"data_stored":65582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":7474,"internal_metadata":27583182},"hb_peers":[0,1,2,3,4,5,7,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509445,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":27152,"kb_used_data":192,"kb_used_omap":7,"kb_used_meta":26936,"kb_avail":104830448,"statfs":{"total":107374182400,"available":107346378752,"internally_reserved":0,"allocated":196608,"data_stored":65582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":7475,"internal_metadata":27583181},"hb_peers":[0,1,2,3,4,6,7,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509444,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":27152,"kb_used_data":192,"kb_used_omap":7,"kb_used_meta":26936,"kb_avail":104830448,"statfs":{"total":107374182400,"available":107346378752,"internally_reserved":0,"allocated":196608,"data_stored":65582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":7475,"internal_metadata":27583181},"hb_peers":[0,1,2,3,5,6,7,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":15,"seq":64424509444,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":27152,"kb_used_data":192,"kb_used_omap":7,"kb_used_meta":26936,"kb_avail":104830448,"statfs":{"total":107374182400,"available":107346378752,"internally_reserved":0,"allocated":196608,"data_stored":65582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":7473,"internal_metadata":27583183},"hb_peers":[0,1,2,4,5,6,7,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":15,"seq":64424509445,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":104857600,"kb_used":27152,"kb_used_data":192,"kb_used_omap":7,"kb_used_meta":26936,"kb_avail":104830448,"statfs":{"total":107374182400,"available":107346378752,"internally_reserved":0,"allocated":196608,"data_stored":65582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":7475,"internal_metadata":27583181},"hb_peers":[0,1,3,4,5,6,7,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":15,"seq":64424509445,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":27152,"kb_used_data":192,"kb_used_omap":7,"kb_used_meta":26936,"kb_avail":104830448,"statfs":{"total":107374182400,"available":107346378752,"internally_reserved":0,"allocated":196608,"data_stored":65582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":7475,"internal_metadata":27583181},"hb_peers":[0,2,3,4,5,6,7,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":15,"seq":64424509445,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":27156,"kb_used_data":196,"kb_used_omap":7,"kb_used_meta":26936,"kb_avail":104830444,"statfs":{"total":107374182400,"available":107346374656,"internally_reserved":0,"allocated":200704,"data_stored":65601,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":7475,"internal_metadata":27583181},"hb_peers":[1,2,3,4,5,6,7,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T23:07:20.009 INFO:tasks.ceph.ceph_manager.ceph:clean! 2026-02-21T23:07:20.009 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2026-02-21T23:07:20.009 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2026-02-21T23:07:20.009 DEBUG:teuthology.orchestra.run.trial150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-21T23:07:20.211 INFO:teuthology.orchestra.run.trial150.stdout: 2026-02-21T23:07:20.212 INFO:teuthology.orchestra.run.trial150.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2026-02-21T23:07:20.220 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2026-02-21T23:07:20.220 INFO:teuthology.run_tasks:Running task workunit... 2026-02-21T23:07:20.234 INFO:tasks.workunit:Pulling workunits from ref 53f8441172c46e34523029ab88f9db7a45514e84 2026-02-21T23:07:20.234 INFO:tasks.workunit:Making a separate scratch dir for every client... 2026-02-21T23:07:20.234 INFO:tasks.workunit:timeout=3h 2026-02-21T23:07:20.234 INFO:tasks.workunit:cleanup=True 2026-02-21T23:07:20.234 DEBUG:teuthology.orchestra.run.trial150:> stat -- /home/ubuntu/cephtest/mnt.0 2026-02-21T23:07:20.255 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-21T23:07:20.255 INFO:teuthology.orchestra.run.trial150.stderr:stat: cannot statx '/home/ubuntu/cephtest/mnt.0': No such file or directory 2026-02-21T23:07:20.255 DEBUG:teuthology.orchestra.run.trial150:> mkdir -- /home/ubuntu/cephtest/mnt.0 2026-02-21T23:07:20.312 INFO:tasks.workunit:Created dir /home/ubuntu/cephtest/mnt.0 2026-02-21T23:07:20.313 DEBUG:teuthology.orchestra.run.trial150:> cd -- /home/ubuntu/cephtest/mnt.0 && mkdir -- client.0 2026-02-21T23:07:20.369 DEBUG:teuthology.orchestra.run.trial150:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://github.com/NitzanMordhai/ceph.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 53f8441172c46e34523029ab88f9db7a45514e84 2026-02-21T23:07:20.432 INFO:tasks.workunit.client.0.trial150.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2026-02-21T23:07:43.058 INFO:tasks.workunit.client.0.trial150.stderr:Note: switching to '53f8441172c46e34523029ab88f9db7a45514e84'. 2026-02-21T23:07:43.058 INFO:tasks.workunit.client.0.trial150.stderr: 2026-02-21T23:07:43.058 INFO:tasks.workunit.client.0.trial150.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2026-02-21T23:07:43.058 INFO:tasks.workunit.client.0.trial150.stderr:changes and commit them, and you can discard any commits you make in this 2026-02-21T23:07:43.058 INFO:tasks.workunit.client.0.trial150.stderr:state without impacting any branches by switching back to a branch. 2026-02-21T23:07:43.059 INFO:tasks.workunit.client.0.trial150.stderr: 2026-02-21T23:07:43.059 INFO:tasks.workunit.client.0.trial150.stderr:If you want to create a new branch to retain commits you create, you may 2026-02-21T23:07:43.059 INFO:tasks.workunit.client.0.trial150.stderr:do so (now or later) by using -c with the switch command. Example: 2026-02-21T23:07:43.059 INFO:tasks.workunit.client.0.trial150.stderr: 2026-02-21T23:07:43.059 INFO:tasks.workunit.client.0.trial150.stderr: git switch -c 2026-02-21T23:07:43.059 INFO:tasks.workunit.client.0.trial150.stderr: 2026-02-21T23:07:43.059 INFO:tasks.workunit.client.0.trial150.stderr:Or undo this operation with: 2026-02-21T23:07:43.059 INFO:tasks.workunit.client.0.trial150.stderr: 2026-02-21T23:07:43.059 INFO:tasks.workunit.client.0.trial150.stderr: git switch - 2026-02-21T23:07:43.059 INFO:tasks.workunit.client.0.trial150.stderr: 2026-02-21T23:07:43.060 INFO:tasks.workunit.client.0.trial150.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2026-02-21T23:07:43.060 INFO:tasks.workunit.client.0.trial150.stderr: 2026-02-21T23:07:43.060 INFO:tasks.workunit.client.0.trial150.stderr:HEAD is now at 53f8441172c Merge pull request #67221 from guits/node-proxy-various-fixes 2026-02-21T23:07:43.061 DEBUG:teuthology.orchestra.run.trial150:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2026-02-21T23:07:43.119 INFO:tasks.workunit.client.0.trial150.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2026-02-21T23:07:43.121 INFO:tasks.workunit.client.0.trial150.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2026-02-21T23:07:43.121 INFO:tasks.workunit.client.0.trial150.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2026-02-21T23:07:43.209 INFO:tasks.workunit.client.0.trial150.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2026-02-21T23:07:43.240 INFO:tasks.workunit.client.0.trial150.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2026-02-21T23:07:43.265 INFO:tasks.workunit.client.0.trial150.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2026-02-21T23:07:43.267 INFO:tasks.workunit.client.0.trial150.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2026-02-21T23:07:43.267 INFO:tasks.workunit.client.0.trial150.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2026-02-21T23:07:43.289 INFO:tasks.workunit.client.0.trial150.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2026-02-21T23:07:43.294 DEBUG:teuthology.orchestra.run.trial150:> set -ex 2026-02-21T23:07:43.294 DEBUG:teuthology.orchestra.run.trial150:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2026-02-21T23:07:43.359 INFO:tasks.workunit:Running workunits matching rados/test_health_warnings.sh on client.0... 2026-02-21T23:07:43.360 INFO:tasks.workunit:Running workunit rados/test_health_warnings.sh... 2026-02-21T23:07:43.360 DEBUG:teuthology.orchestra.run.trial150:workunit test rados/test_health_warnings.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=53f8441172c46e34523029ab88f9db7a45514e84 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rados/test_health_warnings.sh 2026-02-21T23:07:43.420 INFO:tasks.workunit.client.0.trial150.stderr:+ crushtool -o crushmap --build --num_osds 10 host straw 2 rack straw 2 row straw 2 root straw 0 2026-02-21T23:07:43.429 INFO:tasks.workunit.client.0.trial150.stderr:+ ceph osd setcrushmap -i crushmap 2026-02-21T23:07:44.965 INFO:tasks.workunit.client.0.trial150.stderr:6 2026-02-21T23:07:44.975 INFO:tasks.workunit.client.0.trial150.stderr:+ ceph osd tree 2026-02-21T23:07:45.189 INFO:tasks.workunit.client.0.trial150.stdout:ID CLASS WEIGHT TYPE NAME STATUS REWEIGHT PRI-AFF 2026-02-21T23:07:45.190 INFO:tasks.workunit.client.0.trial150.stdout:-11 10.00000 root root 2026-02-21T23:07:45.190 INFO:tasks.workunit.client.0.trial150.stdout: -9 8.00000 row row0 2026-02-21T23:07:45.190 INFO:tasks.workunit.client.0.trial150.stdout: -6 4.00000 rack rack0 2026-02-21T23:07:45.190 INFO:tasks.workunit.client.0.trial150.stdout: -1 2.00000 host host0 2026-02-21T23:07:45.190 INFO:tasks.workunit.client.0.trial150.stdout: 0 1.00000 osd.0 up 1.00000 1.00000 2026-02-21T23:07:45.190 INFO:tasks.workunit.client.0.trial150.stdout: 1 1.00000 osd.1 up 1.00000 1.00000 2026-02-21T23:07:45.190 INFO:tasks.workunit.client.0.trial150.stdout: -2 2.00000 host host1 2026-02-21T23:07:45.190 INFO:tasks.workunit.client.0.trial150.stdout: 2 1.00000 osd.2 up 1.00000 1.00000 2026-02-21T23:07:45.190 INFO:tasks.workunit.client.0.trial150.stdout: 3 1.00000 osd.3 up 1.00000 1.00000 2026-02-21T23:07:45.190 INFO:tasks.workunit.client.0.trial150.stdout: -7 4.00000 rack rack1 2026-02-21T23:07:45.190 INFO:tasks.workunit.client.0.trial150.stdout: -3 2.00000 host host2 2026-02-21T23:07:45.190 INFO:tasks.workunit.client.0.trial150.stdout: 4 1.00000 osd.4 up 1.00000 1.00000 2026-02-21T23:07:45.190 INFO:tasks.workunit.client.0.trial150.stdout: 5 1.00000 osd.5 up 1.00000 1.00000 2026-02-21T23:07:45.190 INFO:tasks.workunit.client.0.trial150.stdout: -4 2.00000 host host3 2026-02-21T23:07:45.190 INFO:tasks.workunit.client.0.trial150.stdout: 6 1.00000 osd.6 up 1.00000 1.00000 2026-02-21T23:07:45.190 INFO:tasks.workunit.client.0.trial150.stdout: 7 1.00000 osd.7 up 1.00000 1.00000 2026-02-21T23:07:45.190 INFO:tasks.workunit.client.0.trial150.stdout:-10 2.00000 row row1 2026-02-21T23:07:45.190 INFO:tasks.workunit.client.0.trial150.stdout: -8 2.00000 rack rack2 2026-02-21T23:07:45.190 INFO:tasks.workunit.client.0.trial150.stdout: -5 2.00000 host host4 2026-02-21T23:07:45.190 INFO:tasks.workunit.client.0.trial150.stdout: 8 1.00000 osd.8 up 1.00000 1.00000 2026-02-21T23:07:45.190 INFO:tasks.workunit.client.0.trial150.stdout: 9 1.00000 osd.9 up 1.00000 1.00000 2026-02-21T23:07:45.197 INFO:tasks.workunit.client.0.trial150.stderr:+ ceph tell 'osd.*' injectargs --osd_max_markdown_count 1024 --osd_max_markdown_period 1 2026-02-21T23:07:45.262 INFO:tasks.workunit.client.0.trial150.stdout:osd.0: {} 2026-02-21T23:07:45.278 INFO:tasks.workunit.client.0.trial150.stdout:osd.1: {} 2026-02-21T23:07:45.295 INFO:tasks.workunit.client.0.trial150.stdout:osd.2: {} 2026-02-21T23:07:45.311 INFO:tasks.workunit.client.0.trial150.stdout:osd.3: {} 2026-02-21T23:07:45.329 INFO:tasks.workunit.client.0.trial150.stdout:osd.4: {} 2026-02-21T23:07:45.347 INFO:tasks.workunit.client.0.trial150.stdout:osd.5: {} 2026-02-21T23:07:45.364 INFO:tasks.workunit.client.0.trial150.stdout:osd.6: {} 2026-02-21T23:07:45.380 INFO:tasks.workunit.client.0.trial150.stdout:osd.7: {} 2026-02-21T23:07:45.396 INFO:tasks.workunit.client.0.trial150.stdout:osd.8: {} 2026-02-21T23:07:45.413 INFO:tasks.workunit.client.0.trial150.stdout:osd.9: {} 2026-02-21T23:07:45.419 INFO:tasks.workunit.client.0.trial150.stderr:+ ceph osd set noout 2026-02-21T23:07:46.968 INFO:tasks.workunit.client.0.trial150.stderr:noout is set 2026-02-21T23:07:46.977 INFO:tasks.workunit.client.0.trial150.stderr:+ test_mark_two_osds_same_host_down 2026-02-21T23:07:46.977 INFO:tasks.workunit.client.0.trial150.stderr:+ ceph osd set noup 2026-02-21T23:07:48.971 INFO:tasks.workunit.client.0.trial150.stderr:noup is set 2026-02-21T23:07:48.980 INFO:tasks.workunit.client.0.trial150.stderr:+ ceph osd down osd.0 osd.1 2026-02-21T23:07:49.141 INFO:tasks.ceph.mon.a.trial150.stderr:2026-02-21T23:07:49.141+0000 7fae333f1640 -1 mon.a@0(leader).osd e27 definitely_dead 0 2026-02-21T23:07:50.034 INFO:tasks.ceph.mon.a.trial150.stderr:2026-02-21T23:07:50.033+0000 7fae333f1640 -1 mon.a@0(leader).osd e28 definitely_dead 0 2026-02-21T23:07:50.034 INFO:tasks.workunit.client.0.trial150.stderr:osd.0 is already down. osd.1 is already down. 2026-02-21T23:07:50.042 INFO:tasks.workunit.client.0.trial150.stderr:+ ceph health detail 2026-02-21T23:07:50.335 INFO:tasks.workunit.client.0.trial150.stdout:HEALTH_WARN noup,noout flag(s) set; 2 osds down; 1 host (2 osds) down 2026-02-21T23:07:50.335 INFO:tasks.workunit.client.0.trial150.stdout:[WRN] OSDMAP_FLAGS: noup,noout flag(s) set 2026-02-21T23:07:50.335 INFO:tasks.workunit.client.0.trial150.stdout:[WRN] OSD_DOWN: 2 osds down 2026-02-21T23:07:50.335 INFO:tasks.workunit.client.0.trial150.stdout: osd.0 (root=root,row=row0,rack=rack0,host=host0) is down 2026-02-21T23:07:50.335 INFO:tasks.workunit.client.0.trial150.stdout: osd.1 (root=root,row=row0,rack=rack0,host=host0) is down 2026-02-21T23:07:50.335 INFO:tasks.workunit.client.0.trial150.stdout:[WRN] OSD_HOST_DOWN: 1 host (2 osds) down 2026-02-21T23:07:50.335 INFO:tasks.workunit.client.0.trial150.stdout: host host0 (root=root,row=row0,rack=rack0) (2 osds) is down 2026-02-21T23:07:50.343 INFO:tasks.workunit.client.0.trial150.stderr:+ ceph health 2026-02-21T23:07:50.343 INFO:tasks.workunit.client.0.trial150.stderr:+ grep '1 host' 2026-02-21T23:07:50.622 INFO:tasks.workunit.client.0.trial150.stdout:HEALTH_WARN noup,noout flag(s) set; 2 osds down; 1 host (2 osds) down 2026-02-21T23:07:50.623 INFO:tasks.workunit.client.0.trial150.stderr:+ ceph health 2026-02-21T23:07:50.624 INFO:tasks.workunit.client.0.trial150.stderr:+ grep '2 osds' 2026-02-21T23:07:50.896 INFO:tasks.workunit.client.0.trial150.stdout:HEALTH_WARN noup,noout flag(s) set; 2 osds down; 1 host (2 osds) down 2026-02-21T23:07:50.897 INFO:tasks.workunit.client.0.trial150.stderr:+ ceph health detail 2026-02-21T23:07:50.897 INFO:tasks.workunit.client.0.trial150.stderr:+ grep osd.0 2026-02-21T23:07:51.199 INFO:tasks.workunit.client.0.trial150.stdout: osd.0 (root=root,row=row0,rack=rack0,host=host0) is down 2026-02-21T23:07:51.200 INFO:tasks.workunit.client.0.trial150.stderr:+ ceph health detail 2026-02-21T23:07:51.200 INFO:tasks.workunit.client.0.trial150.stderr:+ grep osd.1 2026-02-21T23:07:51.505 INFO:tasks.workunit.client.0.trial150.stdout: osd.1 (root=root,row=row0,rack=rack0,host=host0) is down 2026-02-21T23:07:51.505 INFO:tasks.workunit.client.0.trial150.stderr:+ ceph osd unset noup 2026-02-21T23:07:52.984 INFO:tasks.workunit.client.0.trial150.stderr:noup is unset 2026-02-21T23:07:52.994 INFO:tasks.workunit.client.0.trial150.stderr:+ wait_for_healthy 2026-02-21T23:07:52.994 INFO:tasks.workunit.client.0.trial150.stderr:+ ceph health 2026-02-21T23:07:52.994 INFO:tasks.workunit.client.0.trial150.stderr:+ grep down 2026-02-21T23:07:53.136 INFO:tasks.ceph.osd.0.trial150.stderr:2026-02-21T23:07:53.136+0000 7f64c352a640 -1 osd.0 31 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2026-02-21T23:07:53.265 INFO:tasks.workunit.client.0.trial150.stdout:HEALTH_WARN noout flag(s) set; 2 osds down; 1 host (2 osds) down; Degraded data redundancy: 1/4 objects degraded (25.000%), 1 pg degraded 2026-02-21T23:07:53.266 INFO:tasks.workunit.client.0.trial150.stderr:+ sleep 1 2026-02-21T23:07:54.267 INFO:tasks.workunit.client.0.trial150.stderr:+ ceph health 2026-02-21T23:07:54.267 INFO:tasks.workunit.client.0.trial150.stderr:+ grep down 2026-02-21T23:07:54.547 INFO:tasks.workunit.client.0.trial150.stdout:HEALTH_WARN noout flag(s) set; 1 osds down; Degraded data redundancy: 1/4 objects degraded (25.000%), 1 pg degraded 2026-02-21T23:07:54.547 INFO:tasks.workunit.client.0.trial150.stderr:+ sleep 1 2026-02-21T23:07:54.553 INFO:tasks.ceph.osd.1.trial150.stderr:2026-02-21T23:07:54.553+0000 7ffaf5bef640 -1 osd.1 32 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2026-02-21T23:07:55.548 INFO:tasks.workunit.client.0.trial150.stderr:+ ceph health 2026-02-21T23:07:55.548 INFO:tasks.workunit.client.0.trial150.stderr:+ grep down 2026-02-21T23:07:55.820 INFO:tasks.workunit.client.0.trial150.stderr:+ test_mark_two_osds_same_rack_down 2026-02-21T23:07:55.821 INFO:tasks.workunit.client.0.trial150.stderr:+ ceph osd set noup 2026-02-21T23:07:56.993 INFO:tasks.workunit.client.0.trial150.stderr:noup is set 2026-02-21T23:07:57.002 INFO:tasks.workunit.client.0.trial150.stderr:+ ceph osd down osd.8 osd.9 2026-02-21T23:07:57.162 INFO:tasks.ceph.mon.a.trial150.stderr:2026-02-21T23:07:57.162+0000 7fae333f1640 -1 mon.a@0(leader).osd e35 definitely_dead 0 2026-02-21T23:07:58.057 INFO:tasks.ceph.mon.a.trial150.stderr:2026-02-21T23:07:58.056+0000 7fae333f1640 -1 mon.a@0(leader).osd e36 definitely_dead 0 2026-02-21T23:07:58.057 INFO:tasks.workunit.client.0.trial150.stderr:osd.8 is already down. osd.9 is already down. 2026-02-21T23:07:58.065 INFO:tasks.workunit.client.0.trial150.stderr:+ ceph health detail 2026-02-21T23:07:58.358 INFO:tasks.workunit.client.0.trial150.stdout:HEALTH_WARN noup,noout flag(s) set; 2 osds down; 1 host (2 osds) down; 1 rack (2 osds) down; 1 row (2 osds) down; Degraded data redundancy: 1/4 objects degraded (25.000%), 1 pg degraded 2026-02-21T23:07:58.359 INFO:tasks.workunit.client.0.trial150.stdout:[WRN] OSDMAP_FLAGS: noup,noout flag(s) set 2026-02-21T23:07:58.359 INFO:tasks.workunit.client.0.trial150.stdout:[WRN] OSD_DOWN: 2 osds down 2026-02-21T23:07:58.359 INFO:tasks.workunit.client.0.trial150.stdout: osd.8 (root=root,row=row1,rack=rack2,host=host4) is down 2026-02-21T23:07:58.359 INFO:tasks.workunit.client.0.trial150.stdout: osd.9 (root=root,row=row1,rack=rack2,host=host4) is down 2026-02-21T23:07:58.359 INFO:tasks.workunit.client.0.trial150.stdout:[WRN] OSD_HOST_DOWN: 1 host (2 osds) down 2026-02-21T23:07:58.359 INFO:tasks.workunit.client.0.trial150.stdout: host host4 (root=root,row=row1,rack=rack2) (2 osds) is down 2026-02-21T23:07:58.359 INFO:tasks.workunit.client.0.trial150.stdout:[WRN] OSD_RACK_DOWN: 1 rack (2 osds) down 2026-02-21T23:07:58.359 INFO:tasks.workunit.client.0.trial150.stdout: rack rack2 (root=root,row=row1) (2 osds) is down 2026-02-21T23:07:58.359 INFO:tasks.workunit.client.0.trial150.stdout:[WRN] OSD_ROW_DOWN: 1 row (2 osds) down 2026-02-21T23:07:58.360 INFO:tasks.workunit.client.0.trial150.stdout: row row1 (root=root) (2 osds) is down 2026-02-21T23:07:58.360 INFO:tasks.workunit.client.0.trial150.stdout:[WRN] PG_DEGRADED: Degraded data redundancy: 1/4 objects degraded (25.000%), 1 pg degraded 2026-02-21T23:07:58.360 INFO:tasks.workunit.client.0.trial150.stdout: pg 1.3 is active+undersized+degraded+wait, acting [3] 2026-02-21T23:07:58.366 INFO:tasks.workunit.client.0.trial150.stderr:+ ceph health 2026-02-21T23:07:58.366 INFO:tasks.workunit.client.0.trial150.stderr:+ grep '1 host' 2026-02-21T23:07:58.639 INFO:tasks.workunit.client.0.trial150.stdout:HEALTH_WARN noup,noout flag(s) set; 2 osds down; 1 host (2 osds) down; 1 rack (2 osds) down; 1 row (2 osds) down; Degraded data redundancy: 1/4 objects degraded (25.000%), 1 pg degraded 2026-02-21T23:07:58.640 INFO:tasks.workunit.client.0.trial150.stderr:+ ceph health 2026-02-21T23:07:58.640 INFO:tasks.workunit.client.0.trial150.stderr:+ grep '1 rack' 2026-02-21T23:07:58.914 INFO:tasks.workunit.client.0.trial150.stdout:HEALTH_WARN noup,noout flag(s) set; 2 osds down; 1 host (2 osds) down; 1 rack (2 osds) down; 1 row (2 osds) down; Degraded data redundancy: 1/4 objects degraded (25.000%), 1 pg degraded 2026-02-21T23:07:58.914 INFO:tasks.workunit.client.0.trial150.stderr:+ ceph health 2026-02-21T23:07:58.914 INFO:tasks.workunit.client.0.trial150.stderr:+ grep '1 row' 2026-02-21T23:07:59.191 INFO:tasks.workunit.client.0.trial150.stdout:HEALTH_WARN noup,noout flag(s) set; 2 osds down; 1 host (2 osds) down; 1 rack (2 osds) down; 1 row (2 osds) down 2026-02-21T23:07:59.192 INFO:tasks.workunit.client.0.trial150.stderr:+ ceph health 2026-02-21T23:07:59.192 INFO:tasks.workunit.client.0.trial150.stderr:+ grep '2 osds' 2026-02-21T23:07:59.467 INFO:tasks.workunit.client.0.trial150.stdout:HEALTH_WARN noup,noout flag(s) set; 2 osds down; 1 host (2 osds) down; 1 rack (2 osds) down; 1 row (2 osds) down 2026-02-21T23:07:59.468 INFO:tasks.workunit.client.0.trial150.stderr:+ ceph health detail 2026-02-21T23:07:59.468 INFO:tasks.workunit.client.0.trial150.stderr:+ grep osd.8 2026-02-21T23:07:59.661 INFO:tasks.ceph.osd.9.trial150.stderr:2026-02-21T23:07:59.661+0000 7f11c93f8640 -1 osd.9 37 osdmap NOUP flag is set, waiting for it to clear 2026-02-21T23:07:59.768 INFO:tasks.workunit.client.0.trial150.stdout: osd.8 (root=root,row=row1,rack=rack2,host=host4) is down 2026-02-21T23:07:59.768 INFO:tasks.workunit.client.0.trial150.stderr:+ ceph health detail 2026-02-21T23:07:59.769 INFO:tasks.workunit.client.0.trial150.stderr:+ grep osd.9 2026-02-21T23:07:59.783 INFO:tasks.ceph.osd.8.trial150.stderr:2026-02-21T23:07:59.784+0000 7f24eaff8640 -1 osd.8 37 osdmap NOUP flag is set, waiting for it to clear 2026-02-21T23:07:59.999 INFO:tasks.ceph.osd.9.trial150.stderr:2026-02-21T23:07:59.998+0000 7f11b896a640 -1 osd.9 38 osdmap NOUP flag is set, waiting for it to clear 2026-02-21T23:07:59.999 INFO:tasks.ceph.osd.8.trial150.stderr:2026-02-21T23:07:59.998+0000 7f24de51c640 -1 osd.8 38 osdmap NOUP flag is set, waiting for it to clear 2026-02-21T23:08:00.071 INFO:tasks.workunit.client.0.trial150.stdout: osd.9 (root=root,row=row1,rack=rack2,host=host4) is down 2026-02-21T23:08:00.071 INFO:tasks.workunit.client.0.trial150.stderr:+ ceph osd unset noup 2026-02-21T23:08:01.001 INFO:tasks.ceph.osd.8.trial150.stderr:2026-02-21T23:08:01.000+0000 7f24e6732640 -1 osd.8 39 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2026-02-21T23:08:01.003 INFO:tasks.ceph.osd.9.trial150.stderr:2026-02-21T23:08:01.003+0000 7f11c0be7640 -1 osd.9 39 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2026-02-21T23:08:02.000 INFO:tasks.workunit.client.0.trial150.stderr:noup is unset 2026-02-21T23:08:02.010 INFO:tasks.workunit.client.0.trial150.stderr:+ wait_for_healthy 2026-02-21T23:08:02.011 INFO:tasks.workunit.client.0.trial150.stderr:+ ceph health 2026-02-21T23:08:02.011 INFO:tasks.workunit.client.0.trial150.stderr:+ grep down 2026-02-21T23:08:02.280 INFO:tasks.workunit.client.0.trial150.stderr:+ test_mark_all_but_last_osds_down 2026-02-21T23:08:02.281 INFO:tasks.workunit.client.0.trial150.stderr:+ ceph osd set noup 2026-02-21T23:08:04.004 INFO:tasks.workunit.client.0.trial150.stderr:noup is set 2026-02-21T23:08:04.014 INFO:tasks.workunit.client.0.trial150.stderr:++ ceph osd ls 2026-02-21T23:08:04.014 INFO:tasks.workunit.client.0.trial150.stderr:++ sed '$d' 2026-02-21T23:08:04.230 INFO:tasks.workunit.client.0.trial150.stderr:+ ceph osd down 0 1 2 3 4 5 6 7 8 2026-02-21T23:08:04.390 INFO:tasks.ceph.mon.a.trial150.stderr:2026-02-21T23:08:04.390+0000 7fae333f1640 -1 mon.a@0(leader).osd e42 definitely_dead 0 2026-02-21T23:08:05.069 INFO:tasks.ceph.mon.a.trial150.stderr:2026-02-21T23:08:05.069+0000 7fae333f1640 -1 mon.a@0(leader).osd e43 definitely_dead 0 2026-02-21T23:08:05.070 INFO:tasks.workunit.client.0.trial150.stderr:osd.0 is already down. osd.1 is already down. osd.2 is already down. osd.3 is already down. osd.4 is already down. osd.5 is already down. osd.6 is already down. osd.7 is already down. osd.8 is already down. 2026-02-21T23:08:05.079 INFO:tasks.workunit.client.0.trial150.stderr:+ ceph health detail 2026-02-21T23:08:05.372 INFO:tasks.workunit.client.0.trial150.stdout:HEALTH_WARN noup,noout flag(s) set; 9 osds down; 4 hosts (8 osds) down; 2 racks (8 osds) down; 1 row (8 osds) down 2026-02-21T23:08:05.372 INFO:tasks.workunit.client.0.trial150.stdout:[WRN] OSDMAP_FLAGS: noup,noout flag(s) set 2026-02-21T23:08:05.372 INFO:tasks.workunit.client.0.trial150.stdout:[WRN] OSD_DOWN: 9 osds down 2026-02-21T23:08:05.372 INFO:tasks.workunit.client.0.trial150.stdout: osd.0 (root=root,row=row0,rack=rack0,host=host0) is down 2026-02-21T23:08:05.372 INFO:tasks.workunit.client.0.trial150.stdout: osd.1 (root=root,row=row0,rack=rack0,host=host0) is down 2026-02-21T23:08:05.372 INFO:tasks.workunit.client.0.trial150.stdout: osd.2 (root=root,row=row0,rack=rack0,host=host1) is down 2026-02-21T23:08:05.372 INFO:tasks.workunit.client.0.trial150.stdout: osd.3 (root=root,row=row0,rack=rack0,host=host1) is down 2026-02-21T23:08:05.372 INFO:tasks.workunit.client.0.trial150.stdout: osd.4 (root=root,row=row0,rack=rack1,host=host2) is down 2026-02-21T23:08:05.372 INFO:tasks.workunit.client.0.trial150.stdout: osd.5 (root=root,row=row0,rack=rack1,host=host2) is down 2026-02-21T23:08:05.372 INFO:tasks.workunit.client.0.trial150.stdout: osd.6 (root=root,row=row0,rack=rack1,host=host3) is down 2026-02-21T23:08:05.372 INFO:tasks.workunit.client.0.trial150.stdout: osd.7 (root=root,row=row0,rack=rack1,host=host3) is down 2026-02-21T23:08:05.373 INFO:tasks.workunit.client.0.trial150.stdout: osd.8 (root=root,row=row1,rack=rack2,host=host4) is down 2026-02-21T23:08:05.373 INFO:tasks.workunit.client.0.trial150.stdout:[WRN] OSD_HOST_DOWN: 4 hosts (8 osds) down 2026-02-21T23:08:05.373 INFO:tasks.workunit.client.0.trial150.stdout: host host0 (root=root,row=row0,rack=rack0) (2 osds) is down 2026-02-21T23:08:05.373 INFO:tasks.workunit.client.0.trial150.stdout: host host1 (root=root,row=row0,rack=rack0) (2 osds) is down 2026-02-21T23:08:05.373 INFO:tasks.workunit.client.0.trial150.stdout: host host2 (root=root,row=row0,rack=rack1) (2 osds) is down 2026-02-21T23:08:05.373 INFO:tasks.workunit.client.0.trial150.stdout: host host3 (root=root,row=row0,rack=rack1) (2 osds) is down 2026-02-21T23:08:05.373 INFO:tasks.workunit.client.0.trial150.stdout:[WRN] OSD_RACK_DOWN: 2 racks (8 osds) down 2026-02-21T23:08:05.373 INFO:tasks.workunit.client.0.trial150.stdout: rack rack0 (root=root,row=row0) (4 osds) is down 2026-02-21T23:08:05.373 INFO:tasks.workunit.client.0.trial150.stdout: rack rack1 (root=root,row=row0) (4 osds) is down 2026-02-21T23:08:05.373 INFO:tasks.workunit.client.0.trial150.stdout:[WRN] OSD_ROW_DOWN: 1 row (8 osds) down 2026-02-21T23:08:05.373 INFO:tasks.workunit.client.0.trial150.stdout: row row0 (root=root) (8 osds) is down 2026-02-21T23:08:05.381 INFO:tasks.workunit.client.0.trial150.stderr:+ ceph health 2026-02-21T23:08:05.381 INFO:tasks.workunit.client.0.trial150.stderr:+ grep '1 row' 2026-02-21T23:08:05.651 INFO:tasks.workunit.client.0.trial150.stdout:HEALTH_WARN noup,noout flag(s) set; 9 osds down; 4 hosts (8 osds) down; 2 racks (8 osds) down; 1 row (8 osds) down 2026-02-21T23:08:05.652 INFO:tasks.workunit.client.0.trial150.stderr:+ ceph health 2026-02-21T23:08:05.652 INFO:tasks.workunit.client.0.trial150.stderr:+ grep '2 racks' 2026-02-21T23:08:05.666 INFO:tasks.ceph.osd.6.trial150.stderr:2026-02-21T23:08:05.666+0000 7f5276bf8640 -1 osd.6 43 osdmap NOUP flag is set, waiting for it to clear 2026-02-21T23:08:05.720 INFO:tasks.ceph.osd.1.trial150.stderr:2026-02-21T23:08:05.719+0000 7ffaf9bf7640 -1 osd.1 43 osdmap NOUP flag is set, waiting for it to clear 2026-02-21T23:08:05.857 INFO:tasks.ceph.osd.0.trial150.stderr:2026-02-21T23:08:05.856+0000 7f64cb5f7640 -1 osd.0 43 osdmap NOUP flag is set, waiting for it to clear 2026-02-21T23:08:05.926 INFO:tasks.workunit.client.0.trial150.stdout:HEALTH_WARN noup,noout flag(s) set; 9 osds down; 4 hosts (8 osds) down; 2 racks (8 osds) down; 1 row (8 osds) down 2026-02-21T23:08:05.927 INFO:tasks.workunit.client.0.trial150.stderr:+ ceph health 2026-02-21T23:08:05.927 INFO:tasks.workunit.client.0.trial150.stderr:+ grep '4 hosts' 2026-02-21T23:08:06.010 INFO:tasks.ceph.osd.0.trial150.stderr:2026-02-21T23:08:06.010+0000 7f64bb314640 -1 osd.0 44 osdmap NOUP flag is set, waiting for it to clear 2026-02-21T23:08:06.011 INFO:tasks.ceph.osd.6.trial150.stderr:2026-02-21T23:08:06.010+0000 7f526a1c0640 -1 osd.6 44 osdmap NOUP flag is set, waiting for it to clear 2026-02-21T23:08:06.011 INFO:tasks.ceph.osd.1.trial150.stderr:2026-02-21T23:08:06.010+0000 7ffaed9c0640 -1 osd.1 44 osdmap NOUP flag is set, waiting for it to clear 2026-02-21T23:08:06.200 INFO:tasks.workunit.client.0.trial150.stdout:HEALTH_WARN noup,noout flag(s) set; 9 osds down; 4 hosts (8 osds) down; 2 racks (8 osds) down; 1 row (8 osds) down 2026-02-21T23:08:06.200 INFO:tasks.workunit.client.0.trial150.stderr:+ ceph health 2026-02-21T23:08:06.200 INFO:tasks.workunit.client.0.trial150.stderr:+ grep '9 osds' 2026-02-21T23:08:06.336 INFO:tasks.ceph.osd.4.trial150.stderr:2026-02-21T23:08:06.336+0000 7fea053f8640 -1 osd.4 44 osdmap NOUP flag is set, waiting for it to clear 2026-02-21T23:08:06.437 INFO:tasks.ceph.osd.5.trial150.stderr:2026-02-21T23:08:06.437+0000 7f3b369f8640 -1 osd.5 44 osdmap NOUP flag is set, waiting for it to clear 2026-02-21T23:08:06.467 INFO:tasks.ceph.osd.3.trial150.stderr:2026-02-21T23:08:06.466+0000 7f2184df8640 -1 osd.3 44 osdmap NOUP flag is set, waiting for it to clear 2026-02-21T23:08:06.472 INFO:tasks.workunit.client.0.trial150.stdout:HEALTH_WARN noup,noout flag(s) set; 9 osds down; 4 hosts (8 osds) down; 2 racks (8 osds) down; 1 row (8 osds) down 2026-02-21T23:08:06.472 INFO:tasks.workunit.client.0.trial150.stderr:+ ceph osd unset noup 2026-02-21T23:08:06.480 INFO:tasks.ceph.osd.2.trial150.stderr:2026-02-21T23:08:06.480+0000 7fee941f8640 -1 osd.2 44 osdmap NOUP flag is set, waiting for it to clear 2026-02-21T23:08:06.741 INFO:tasks.ceph.osd.7.trial150.stderr:2026-02-21T23:08:06.741+0000 7f67cfdf8640 -1 osd.7 44 osdmap NOUP flag is set, waiting for it to clear 2026-02-21T23:08:07.015 INFO:tasks.ceph.osd.6.trial150.stderr:2026-02-21T23:08:07.014+0000 7f52723ef640 -1 osd.6 45 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2026-02-21T23:08:07.015 INFO:tasks.ceph.osd.0.trial150.stderr:2026-02-21T23:08:07.014+0000 7f64c352a640 -1 osd.0 45 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2026-02-21T23:08:07.016 INFO:tasks.ceph.osd.1.trial150.stderr:2026-02-21T23:08:07.014+0000 7ffaf5bef640 -1 osd.1 45 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2026-02-21T23:08:07.016 INFO:tasks.ceph.osd.3.trial150.stderr:2026-02-21T23:08:07.014+0000 7f21805ef640 -1 osd.3 45 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2026-02-21T23:08:07.016 INFO:tasks.ceph.osd.5.trial150.stderr:2026-02-21T23:08:07.014+0000 7f3b2e12a640 -1 osd.5 45 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2026-02-21T23:08:07.017 INFO:tasks.ceph.osd.7.trial150.stderr:2026-02-21T23:08:07.014+0000 7f67c75e7640 -1 osd.7 45 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2026-02-21T23:08:07.017 INFO:tasks.ceph.osd.4.trial150.stderr:2026-02-21T23:08:07.015+0000 7fe9fcbe7640 -1 osd.4 45 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2026-02-21T23:08:07.017 INFO:tasks.ceph.osd.2.trial150.stderr:2026-02-21T23:08:07.015+0000 7fee8f9ef640 -1 osd.2 45 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2026-02-21T23:08:08.014 INFO:tasks.workunit.client.0.trial150.stderr:noup is unset 2026-02-21T23:08:08.025 INFO:tasks.workunit.client.0.trial150.stderr:+ wait_for_healthy 2026-02-21T23:08:08.025 INFO:tasks.workunit.client.0.trial150.stderr:+ ceph health 2026-02-21T23:08:08.025 INFO:tasks.workunit.client.0.trial150.stderr:+ grep down 2026-02-21T23:08:08.300 INFO:tasks.workunit.client.0.trial150.stdout:HEALTH_WARN noout flag(s) set; 1 osds down 2026-02-21T23:08:08.300 INFO:tasks.workunit.client.0.trial150.stderr:+ sleep 1 2026-02-21T23:08:08.924 INFO:tasks.ceph.osd.8.trial150.stderr:2026-02-21T23:08:08.924+0000 7f24e6732640 -1 osd.8 46 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2026-02-21T23:08:09.301 INFO:tasks.workunit.client.0.trial150.stderr:+ ceph health 2026-02-21T23:08:09.301 INFO:tasks.workunit.client.0.trial150.stderr:+ grep down 2026-02-21T23:08:09.572 INFO:tasks.workunit.client.0.trial150.stderr:+ test_mark_two_osds_same_host_down_with_classes 2026-02-21T23:08:09.572 INFO:tasks.workunit.client.0.trial150.stderr:+ ceph osd set noup 2026-02-21T23:08:11.021 INFO:tasks.workunit.client.0.trial150.stderr:noup is set 2026-02-21T23:08:11.029 INFO:tasks.workunit.client.0.trial150.stderr:+ ceph osd crush set-device-class ssd osd.0 osd.2 osd.4 osd.6 osd.8 2026-02-21T23:08:13.025 INFO:tasks.workunit.client.0.trial150.stderr:osd.0 already set to class ssd. set-device-class item id 0 name 'osd.0' device_class 'ssd': no change. osd.2 already set to class ssd. set-device-class item id 2 name 'osd.2' device_class 'ssd': no change. osd.4 already set to class ssd. set-device-class item id 4 name 'osd.4' device_class 'ssd': no change. osd.6 already set to class ssd. set-device-class item id 6 name 'osd.6' device_class 'ssd': no change. osd.8 already set to class ssd. set-device-class item id 8 name 'osd.8' device_class 'ssd': no change. set osd(s) to class 'ssd' 2026-02-21T23:08:13.034 INFO:tasks.workunit.client.0.trial150.stderr:+ ceph osd crush set-device-class hdd osd.1 osd.3 osd.5 osd.7 osd.9 2026-02-21T23:08:15.028 INFO:tasks.workunit.client.0.trial150.stderr:osd.1 already set to class hdd. set-device-class item id 1 name 'osd.1' device_class 'hdd': no change. osd.3 already set to class hdd. set-device-class item id 3 name 'osd.3' device_class 'hdd': no change. osd.5 already set to class hdd. set-device-class item id 5 name 'osd.5' device_class 'hdd': no change. osd.7 already set to class hdd. set-device-class item id 7 name 'osd.7' device_class 'hdd': no change. osd.9 already set to class hdd. set-device-class item id 9 name 'osd.9' device_class 'hdd': no change. set osd(s) to class 'hdd' 2026-02-21T23:08:15.038 INFO:tasks.workunit.client.0.trial150.stderr:+ ceph osd down osd.0 osd.1 2026-02-21T23:08:15.200 INFO:tasks.ceph.mon.a.trial150.stderr:2026-02-21T23:08:15.199+0000 7fae333f1640 -1 mon.a@0(leader).osd e53 definitely_dead 0 2026-02-21T23:08:16.093 INFO:tasks.ceph.mon.a.trial150.stderr:2026-02-21T23:08:16.092+0000 7fae333f1640 -1 mon.a@0(leader).osd e54 definitely_dead 0 2026-02-21T23:08:16.112 INFO:tasks.workunit.client.0.trial150.stderr:osd.0 is already down. osd.1 is already down. 2026-02-21T23:08:16.112 INFO:tasks.workunit.client.0.trial150.stderr:+ ceph health detail 2026-02-21T23:08:16.399 INFO:tasks.workunit.client.0.trial150.stdout:HEALTH_WARN noup,noout flag(s) set; 2 osds down; 1 host (2 osds) down 2026-02-21T23:08:16.399 INFO:tasks.workunit.client.0.trial150.stdout:[WRN] OSDMAP_FLAGS: noup,noout flag(s) set 2026-02-21T23:08:16.399 INFO:tasks.workunit.client.0.trial150.stdout:[WRN] OSD_DOWN: 2 osds down 2026-02-21T23:08:16.399 INFO:tasks.workunit.client.0.trial150.stdout: osd.0 (root=root,row=row0,rack=rack0,host=host0) is down 2026-02-21T23:08:16.399 INFO:tasks.workunit.client.0.trial150.stdout: osd.1 (root=root,row=row0,rack=rack0,host=host0) is down 2026-02-21T23:08:16.399 INFO:tasks.workunit.client.0.trial150.stdout:[WRN] OSD_HOST_DOWN: 1 host (2 osds) down 2026-02-21T23:08:16.399 INFO:tasks.workunit.client.0.trial150.stdout: host host0 (root=root,row=row0,rack=rack0) (2 osds) is down 2026-02-21T23:08:16.409 INFO:tasks.workunit.client.0.trial150.stderr:+ ceph health 2026-02-21T23:08:16.409 INFO:tasks.workunit.client.0.trial150.stderr:+ grep '1 host' 2026-02-21T23:08:16.683 INFO:tasks.workunit.client.0.trial150.stdout:HEALTH_WARN noup,noout flag(s) set; 2 osds down; 1 host (2 osds) down 2026-02-21T23:08:16.683 INFO:tasks.workunit.client.0.trial150.stderr:+ ceph health 2026-02-21T23:08:16.683 INFO:tasks.workunit.client.0.trial150.stderr:+ grep '2 osds' 2026-02-21T23:08:16.958 INFO:tasks.workunit.client.0.trial150.stdout:HEALTH_WARN noup,noout flag(s) set; 2 osds down; 1 host (2 osds) down 2026-02-21T23:08:16.958 INFO:tasks.workunit.client.0.trial150.stderr:+ ceph health detail 2026-02-21T23:08:16.958 INFO:tasks.workunit.client.0.trial150.stderr:+ grep osd.0 2026-02-21T23:08:17.261 INFO:tasks.workunit.client.0.trial150.stdout: osd.0 (root=root,row=row0,rack=rack0,host=host0) is down 2026-02-21T23:08:17.261 INFO:tasks.workunit.client.0.trial150.stderr:+ ceph health detail 2026-02-21T23:08:17.261 INFO:tasks.workunit.client.0.trial150.stderr:+ grep osd.1 2026-02-21T23:08:17.562 INFO:tasks.workunit.client.0.trial150.stdout: osd.1 (root=root,row=row0,rack=rack0,host=host0) is down 2026-02-21T23:08:17.562 INFO:tasks.workunit.client.0.trial150.stderr:+ ceph osd unset noup 2026-02-21T23:08:19.008 INFO:tasks.ceph.osd.0.trial150.stderr:2026-02-21T23:08:19.007+0000 7f64c352a640 -1 osd.0 56 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2026-02-21T23:08:19.035 INFO:tasks.workunit.client.0.trial150.stderr:noup is unset 2026-02-21T23:08:19.044 INFO:tasks.workunit.client.0.trial150.stderr:+ wait_for_healthy 2026-02-21T23:08:19.045 INFO:tasks.workunit.client.0.trial150.stderr:+ ceph health 2026-02-21T23:08:19.045 INFO:tasks.workunit.client.0.trial150.stderr:+ grep down 2026-02-21T23:08:19.316 INFO:tasks.workunit.client.0.trial150.stdout:HEALTH_WARN noout flag(s) set; 1 osds down 2026-02-21T23:08:19.317 INFO:tasks.workunit.client.0.trial150.stderr:+ sleep 1 2026-02-21T23:08:19.687 INFO:tasks.ceph.osd.1.trial150.stderr:2026-02-21T23:08:19.686+0000 7ffaf5bef640 -1 osd.1 57 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2026-02-21T23:08:20.318 INFO:tasks.workunit.client.0.trial150.stderr:+ ceph health 2026-02-21T23:08:20.318 INFO:tasks.workunit.client.0.trial150.stderr:+ grep down 2026-02-21T23:08:20.590 INFO:tasks.workunit.client.0.trial150.stderr:+ exit 0 2026-02-21T23:08:20.590 INFO:teuthology.orchestra.run:Running command with timeout 3600 2026-02-21T23:08:20.590 DEBUG:teuthology.orchestra.run.trial150:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2026-02-21T23:08:20.655 INFO:tasks.workunit:Stopping ['rados/test_health_warnings.sh'] on client.0... 2026-02-21T23:08:20.655 DEBUG:teuthology.orchestra.run.trial150:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2026-02-21T23:08:21.044 DEBUG:teuthology.parallel:result is None 2026-02-21T23:08:21.044 DEBUG:teuthology.orchestra.run.trial150:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2026-02-21T23:08:21.070 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2026-02-21T23:08:21.070 DEBUG:teuthology.orchestra.run.trial150:> rmdir -- /home/ubuntu/cephtest/mnt.0 2026-02-21T23:08:21.127 INFO:tasks.workunit:Deleted artificial mount point /home/ubuntu/cephtest/mnt.0/client.0 2026-02-21T23:08:21.127 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2026-02-21T23:08:21.148 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2026-02-21T23:08:21.149 DEBUG:teuthology.orchestra.run.trial150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T23:08:21.341 INFO:teuthology.orchestra.run.trial150.stdout: 2026-02-21T23:08:21.342 INFO:teuthology.orchestra.run.trial150.stderr:dumped all 2026-02-21T23:08:21.350 INFO:teuthology.orchestra.run.trial150.stdout:{"pg_ready":true,"pg_map":{"version":96,"stamp":"2026-02-21T23:08:21.040084+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":10,"num_per_pool_osds":10,"num_per_pool_omap_osds":8,"kb":1048576000,"kb_used":275440,"kb_used_data":5840,"kb_used_omap":235,"kb_used_meta":269204,"kb_avail":1048300560,"statfs":{"total":1073741824000,"available":1073459773440,"internally_reserved":0,"allocated":5980160,"data_stored":2762982,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":241611,"internal_metadata":275664949},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.007211"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":62,"reported_epoch":52,"state":"active+clean","last_fresh":"2026-02-21T23:08:15.632899+0000","last_change":"2026-02-21T23:08:09.021397+0000","last_active":"2026-02-21T23:08:15.632899+0000","last_peered":"2026-02-21T23:08:15.632899+0000","last_clean":"2026-02-21T23:08:15.632899+0000","last_became_active":"2026-02-21T23:08:09.021232+0000","last_became_peered":"2026-02-21T23:08:09.021232+0000","last_unstale":"2026-02-21T23:08:15.632899+0000","last_undegraded":"2026-02-21T23:08:15.632899+0000","last_fullsized":"2026-02-21T23:08:15.632899+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T23:07:12.930820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T23:07:12.930820+0000","last_clean_scrub_stamp":"2026-02-21T23:07:12.930820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T01:01:37.847843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00033767299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":88,"reported_epoch":53,"state":"active+clean","last_fresh":"2026-02-21T23:08:15.632841+0000","last_change":"2026-02-21T23:08:09.021642+0000","last_active":"2026-02-21T23:08:15.632841+0000","last_peered":"2026-02-21T23:08:15.632841+0000","last_clean":"2026-02-21T23:08:15.632841+0000","last_became_active":"2026-02-21T23:08:09.021460+0000","last_became_peered":"2026-02-21T23:08:09.021460+0000","last_unstale":"2026-02-21T23:08:15.632841+0000","last_undegraded":"2026-02-21T23:08:15.632841+0000","last_fullsized":"2026-02-21T23:08:15.632841+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T23:07:12.930820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T23:07:12.930820+0000","last_clean_scrub_stamp":"2026-02-21T23:07:12.930820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T01:01:37.847843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022801700000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":88,"reported_epoch":53,"state":"active+clean","last_fresh":"2026-02-21T23:08:15.904902+0000","last_change":"2026-02-21T23:08:09.152799+0000","last_active":"2026-02-21T23:08:15.904902+0000","last_peered":"2026-02-21T23:08:15.904902+0000","last_clean":"2026-02-21T23:08:15.904902+0000","last_became_active":"2026-02-21T23:08:09.152693+0000","last_became_peered":"2026-02-21T23:08:09.152693+0000","last_unstale":"2026-02-21T23:08:15.904902+0000","last_undegraded":"2026-02-21T23:08:15.904902+0000","last_fullsized":"2026-02-21T23:08:15.904902+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T23:07:12.930820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T23:07:12.930820+0000","last_clean_scrub_stamp":"2026-02-21T23:07:12.930820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:57:13.040966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00029313000000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0],"acting":[9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":66,"reported_epoch":53,"state":"stale+active+clean","last_fresh":"2026-02-21T23:08:15.029819+0000","last_change":"2026-02-21T23:08:09.021356+0000","last_active":"2026-02-21T23:08:15.029819+0000","last_peered":"2026-02-21T23:08:15.029819+0000","last_clean":"2026-02-21T23:08:15.029819+0000","last_became_active":"2026-02-21T23:08:09.021241+0000","last_became_peered":"2026-02-21T23:08:09.021241+0000","last_unstale":"2026-02-21T23:08:18.033451+0000","last_undegraded":"2026-02-21T23:08:15.029819+0000","last_fullsized":"2026-02-21T23:08:15.029819+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T23:07:12.930820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T23:07:12.930820+0000","last_clean_scrub_stamp":"2026-02-21T23:07:12.930820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T01:42:39.532721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.3","version":"18'1","reported_seq":75,"reported_epoch":52,"state":"active+clean","last_fresh":"2026-02-21T23:08:15.632852+0000","last_change":"2026-02-21T23:08:09.021664+0000","last_active":"2026-02-21T23:08:15.632852+0000","last_peered":"2026-02-21T23:08:15.632852+0000","last_clean":"2026-02-21T23:08:15.632852+0000","last_became_active":"2026-02-21T23:08:09.021515+0000","last_became_peered":"2026-02-21T23:08:09.021515+0000","last_unstale":"2026-02-21T23:08:15.632852+0000","last_undegraded":"2026-02-21T23:08:15.632852+0000","last_fullsized":"2026-02-21T23:08:15.632852+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T23:07:12.930820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T23:07:12.930820+0000","last_clean_scrub_stamp":"2026-02-21T23:07:12.930820+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T02:56:28.927064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.2","version":"20'2","reported_seq":80,"reported_epoch":52,"state":"active+clean","last_fresh":"2026-02-21T23:08:15.632892+0000","last_change":"2026-02-21T23:08:09.021669+0000","last_active":"2026-02-21T23:08:15.632892+0000","last_peered":"2026-02-21T23:08:15.632892+0000","last_clean":"2026-02-21T23:08:15.632892+0000","last_became_active":"2026-02-21T23:08:09.021538+0000","last_became_peered":"2026-02-21T23:08:09.021538+0000","last_unstale":"2026-02-21T23:08:15.632892+0000","last_undegraded":"2026-02-21T23:08:15.632892+0000","last_fullsized":"2026-02-21T23:08:15.632892+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T23:07:12.930820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T23:07:12.930820+0000","last_clean_scrub_stamp":"2026-02-21T23:07:12.930820+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:12:38.733731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044592600000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":66,"reported_epoch":52,"state":"active+clean","last_fresh":"2026-02-21T23:08:15.632813+0000","last_change":"2026-02-21T23:08:09.021344+0000","last_active":"2026-02-21T23:08:15.632813+0000","last_peered":"2026-02-21T23:08:15.632813+0000","last_clean":"2026-02-21T23:08:15.632813+0000","last_became_active":"2026-02-21T23:08:09.021252+0000","last_became_peered":"2026-02-21T23:08:09.021252+0000","last_unstale":"2026-02-21T23:08:15.632813+0000","last_undegraded":"2026-02-21T23:08:15.632813+0000","last_fullsized":"2026-02-21T23:08:15.632813+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T23:07:12.930820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T23:07:12.930820+0000","last_clean_scrub_stamp":"2026-02-21T23:07:12.930820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:30:42.606649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00037064500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":74,"reported_epoch":52,"state":"active+clean","last_fresh":"2026-02-21T23:08:15.632841+0000","last_change":"2026-02-21T23:08:09.021203+0000","last_active":"2026-02-21T23:08:15.632841+0000","last_peered":"2026-02-21T23:08:15.632841+0000","last_clean":"2026-02-21T23:08:15.632841+0000","last_became_active":"2026-02-21T23:08:09.021076+0000","last_became_peered":"2026-02-21T23:08:09.021076+0000","last_unstale":"2026-02-21T23:08:15.632841+0000","last_undegraded":"2026-02-21T23:08:15.632841+0000","last_fullsized":"2026-02-21T23:08:15.632841+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T23:07:12.930820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T23:07:12.930820+0000","last_clean_scrub_stamp":"2026-02-21T23:07:12.930820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T00:21:40.020332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9],"acting":[5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":9}],"osd_stats":[{"osd":9,"up_from":40,"seq":171798691856,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":27548,"kb_used_data":588,"kb_used_omap":27,"kb_used_meta":26916,"kb_avail":104830052,"statfs":{"total":107374182400,"available":107345973248,"internally_reserved":0,"allocated":602112,"data_stored":280722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":27746,"internal_metadata":27562910},"hb_peers":[0,1,2,3,4,5,6,7,8],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":47,"seq":201863462929,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":104857600,"kb_used":27548,"kb_used_data":588,"kb_used_omap":23,"kb_used_meta":26920,"kb_avail":104830052,"statfs":{"total":107374182400,"available":107345973248,"internally_reserved":0,"allocated":602112,"data_stored":280722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":23836,"internal_metadata":27566820},"hb_peers":[0,1,2,3,4,5,6,7,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":46,"seq":197568495633,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":27536,"kb_used_data":576,"kb_used_omap":11,"kb_used_meta":26932,"kb_avail":104830064,"statfs":{"total":107374182400,"available":107345985536,"internally_reserved":0,"allocated":589824,"data_stored":270320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12113,"internal_metadata":27578543},"hb_peers":[0,1,2,3,4,5,6,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495632,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":27548,"kb_used_data":588,"kb_used_omap":25,"kb_used_meta":26918,"kb_avail":104830052,"statfs":{"total":107374182400,"available":107345973248,"internally_reserved":0,"allocated":602112,"data_stored":280722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":26439,"internal_metadata":27564217},"hb_peers":[0,1,2,3,4,5,7,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":46,"seq":197568495633,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":27536,"kb_used_data":576,"kb_used_omap":24,"kb_used_meta":26919,"kb_avail":104830064,"statfs":{"total":107374182400,"available":107345985536,"internally_reserved":0,"allocated":589824,"data_stored":270320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":25138,"internal_metadata":27565518},"hb_peers":[0,1,2,3,4,6,7,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":46,"seq":197568495632,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":27540,"kb_used_data":580,"kb_used_omap":25,"kb_used_meta":26918,"kb_avail":104830060,"statfs":{"total":107374182400,"available":107345981440,"internally_reserved":0,"allocated":593920,"data_stored":270339,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":26440,"internal_metadata":27564216},"hb_peers":[0,1,2,3,5,6,7,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":46,"seq":197568495632,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":27536,"kb_used_data":576,"kb_used_omap":22,"kb_used_meta":26921,"kb_avail":104830064,"statfs":{"total":107374182400,"available":107345985536,"internally_reserved":0,"allocated":589824,"data_stored":270320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":23183,"internal_metadata":27567473},"hb_peers":[0,1,2,4,5,6,7,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":46,"seq":197568495633,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":104857600,"kb_used":27560,"kb_used_data":600,"kb_used_omap":25,"kb_used_meta":26918,"kb_avail":104830040,"statfs":{"total":107374182400,"available":107345960960,"internally_reserved":0,"allocated":614400,"data_stored":288456,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":26440,"internal_metadata":27564216},"hb_peers":[0,1,3,4,5,6,7,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":46,"seq":197568495633,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":27552,"kb_used_data":592,"kb_used_omap":23,"kb_used_meta":26920,"kb_avail":104830048,"statfs":{"total":107374182400,"available":107345969152,"internally_reserved":0,"allocated":606208,"data_stored":280741,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":24487,"internal_metadata":27566169},"hb_peers":[0,2,3,4,5,6,7,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":46,"seq":197568495633,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":27536,"kb_used_data":576,"kb_used_omap":25,"kb_used_meta":26918,"kb_avail":104830064,"statfs":{"total":107374182400,"available":107345985536,"internally_reserved":0,"allocated":589824,"data_stored":270320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":25789,"internal_metadata":27564867},"hb_peers":[1,2,3,4,5,6,7,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T23:08:21.352 DEBUG:teuthology.orchestra.run.trial150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T23:08:21.533 INFO:teuthology.orchestra.run.trial150.stdout: 2026-02-21T23:08:21.534 INFO:teuthology.orchestra.run.trial150.stderr:dumped all 2026-02-21T23:08:21.543 INFO:teuthology.orchestra.run.trial150.stdout:{"pg_ready":true,"pg_map":{"version":96,"stamp":"2026-02-21T23:08:21.040084+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":10,"num_per_pool_osds":10,"num_per_pool_omap_osds":8,"kb":1048576000,"kb_used":275440,"kb_used_data":5840,"kb_used_omap":235,"kb_used_meta":269204,"kb_avail":1048300560,"statfs":{"total":1073741824000,"available":1073459773440,"internally_reserved":0,"allocated":5980160,"data_stored":2762982,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":241611,"internal_metadata":275664949},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.007211"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":62,"reported_epoch":52,"state":"active+clean","last_fresh":"2026-02-21T23:08:15.632899+0000","last_change":"2026-02-21T23:08:09.021397+0000","last_active":"2026-02-21T23:08:15.632899+0000","last_peered":"2026-02-21T23:08:15.632899+0000","last_clean":"2026-02-21T23:08:15.632899+0000","last_became_active":"2026-02-21T23:08:09.021232+0000","last_became_peered":"2026-02-21T23:08:09.021232+0000","last_unstale":"2026-02-21T23:08:15.632899+0000","last_undegraded":"2026-02-21T23:08:15.632899+0000","last_fullsized":"2026-02-21T23:08:15.632899+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T23:07:12.930820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T23:07:12.930820+0000","last_clean_scrub_stamp":"2026-02-21T23:07:12.930820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T01:01:37.847843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00033767299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":88,"reported_epoch":53,"state":"active+clean","last_fresh":"2026-02-21T23:08:15.632841+0000","last_change":"2026-02-21T23:08:09.021642+0000","last_active":"2026-02-21T23:08:15.632841+0000","last_peered":"2026-02-21T23:08:15.632841+0000","last_clean":"2026-02-21T23:08:15.632841+0000","last_became_active":"2026-02-21T23:08:09.021460+0000","last_became_peered":"2026-02-21T23:08:09.021460+0000","last_unstale":"2026-02-21T23:08:15.632841+0000","last_undegraded":"2026-02-21T23:08:15.632841+0000","last_fullsized":"2026-02-21T23:08:15.632841+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T23:07:12.930820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T23:07:12.930820+0000","last_clean_scrub_stamp":"2026-02-21T23:07:12.930820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T01:01:37.847843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022801700000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":88,"reported_epoch":53,"state":"active+clean","last_fresh":"2026-02-21T23:08:15.904902+0000","last_change":"2026-02-21T23:08:09.152799+0000","last_active":"2026-02-21T23:08:15.904902+0000","last_peered":"2026-02-21T23:08:15.904902+0000","last_clean":"2026-02-21T23:08:15.904902+0000","last_became_active":"2026-02-21T23:08:09.152693+0000","last_became_peered":"2026-02-21T23:08:09.152693+0000","last_unstale":"2026-02-21T23:08:15.904902+0000","last_undegraded":"2026-02-21T23:08:15.904902+0000","last_fullsized":"2026-02-21T23:08:15.904902+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T23:07:12.930820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T23:07:12.930820+0000","last_clean_scrub_stamp":"2026-02-21T23:07:12.930820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:57:13.040966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00029313000000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0],"acting":[9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":66,"reported_epoch":53,"state":"stale+active+clean","last_fresh":"2026-02-21T23:08:15.029819+0000","last_change":"2026-02-21T23:08:09.021356+0000","last_active":"2026-02-21T23:08:15.029819+0000","last_peered":"2026-02-21T23:08:15.029819+0000","last_clean":"2026-02-21T23:08:15.029819+0000","last_became_active":"2026-02-21T23:08:09.021241+0000","last_became_peered":"2026-02-21T23:08:09.021241+0000","last_unstale":"2026-02-21T23:08:18.033451+0000","last_undegraded":"2026-02-21T23:08:15.029819+0000","last_fullsized":"2026-02-21T23:08:15.029819+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T23:07:12.930820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T23:07:12.930820+0000","last_clean_scrub_stamp":"2026-02-21T23:07:12.930820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T01:42:39.532721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.3","version":"18'1","reported_seq":75,"reported_epoch":52,"state":"active+clean","last_fresh":"2026-02-21T23:08:15.632852+0000","last_change":"2026-02-21T23:08:09.021664+0000","last_active":"2026-02-21T23:08:15.632852+0000","last_peered":"2026-02-21T23:08:15.632852+0000","last_clean":"2026-02-21T23:08:15.632852+0000","last_became_active":"2026-02-21T23:08:09.021515+0000","last_became_peered":"2026-02-21T23:08:09.021515+0000","last_unstale":"2026-02-21T23:08:15.632852+0000","last_undegraded":"2026-02-21T23:08:15.632852+0000","last_fullsized":"2026-02-21T23:08:15.632852+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T23:07:12.930820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T23:07:12.930820+0000","last_clean_scrub_stamp":"2026-02-21T23:07:12.930820+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T02:56:28.927064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.2","version":"20'2","reported_seq":80,"reported_epoch":52,"state":"active+clean","last_fresh":"2026-02-21T23:08:15.632892+0000","last_change":"2026-02-21T23:08:09.021669+0000","last_active":"2026-02-21T23:08:15.632892+0000","last_peered":"2026-02-21T23:08:15.632892+0000","last_clean":"2026-02-21T23:08:15.632892+0000","last_became_active":"2026-02-21T23:08:09.021538+0000","last_became_peered":"2026-02-21T23:08:09.021538+0000","last_unstale":"2026-02-21T23:08:15.632892+0000","last_undegraded":"2026-02-21T23:08:15.632892+0000","last_fullsized":"2026-02-21T23:08:15.632892+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T23:07:12.930820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T23:07:12.930820+0000","last_clean_scrub_stamp":"2026-02-21T23:07:12.930820+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:12:38.733731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044592600000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":66,"reported_epoch":52,"state":"active+clean","last_fresh":"2026-02-21T23:08:15.632813+0000","last_change":"2026-02-21T23:08:09.021344+0000","last_active":"2026-02-21T23:08:15.632813+0000","last_peered":"2026-02-21T23:08:15.632813+0000","last_clean":"2026-02-21T23:08:15.632813+0000","last_became_active":"2026-02-21T23:08:09.021252+0000","last_became_peered":"2026-02-21T23:08:09.021252+0000","last_unstale":"2026-02-21T23:08:15.632813+0000","last_undegraded":"2026-02-21T23:08:15.632813+0000","last_fullsized":"2026-02-21T23:08:15.632813+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T23:07:12.930820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T23:07:12.930820+0000","last_clean_scrub_stamp":"2026-02-21T23:07:12.930820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:30:42.606649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00037064500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":74,"reported_epoch":52,"state":"active+clean","last_fresh":"2026-02-21T23:08:15.632841+0000","last_change":"2026-02-21T23:08:09.021203+0000","last_active":"2026-02-21T23:08:15.632841+0000","last_peered":"2026-02-21T23:08:15.632841+0000","last_clean":"2026-02-21T23:08:15.632841+0000","last_became_active":"2026-02-21T23:08:09.021076+0000","last_became_peered":"2026-02-21T23:08:09.021076+0000","last_unstale":"2026-02-21T23:08:15.632841+0000","last_undegraded":"2026-02-21T23:08:15.632841+0000","last_fullsized":"2026-02-21T23:08:15.632841+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T23:07:12.930820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T23:07:12.930820+0000","last_clean_scrub_stamp":"2026-02-21T23:07:12.930820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T00:21:40.020332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9],"acting":[5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":9}],"osd_stats":[{"osd":9,"up_from":40,"seq":171798691856,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":27548,"kb_used_data":588,"kb_used_omap":27,"kb_used_meta":26916,"kb_avail":104830052,"statfs":{"total":107374182400,"available":107345973248,"internally_reserved":0,"allocated":602112,"data_stored":280722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":27746,"internal_metadata":27562910},"hb_peers":[0,1,2,3,4,5,6,7,8],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":47,"seq":201863462929,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":104857600,"kb_used":27548,"kb_used_data":588,"kb_used_omap":23,"kb_used_meta":26920,"kb_avail":104830052,"statfs":{"total":107374182400,"available":107345973248,"internally_reserved":0,"allocated":602112,"data_stored":280722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":23836,"internal_metadata":27566820},"hb_peers":[0,1,2,3,4,5,6,7,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":46,"seq":197568495633,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":27536,"kb_used_data":576,"kb_used_omap":11,"kb_used_meta":26932,"kb_avail":104830064,"statfs":{"total":107374182400,"available":107345985536,"internally_reserved":0,"allocated":589824,"data_stored":270320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12113,"internal_metadata":27578543},"hb_peers":[0,1,2,3,4,5,6,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495632,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":27548,"kb_used_data":588,"kb_used_omap":25,"kb_used_meta":26918,"kb_avail":104830052,"statfs":{"total":107374182400,"available":107345973248,"internally_reserved":0,"allocated":602112,"data_stored":280722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":26439,"internal_metadata":27564217},"hb_peers":[0,1,2,3,4,5,7,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":46,"seq":197568495633,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":27536,"kb_used_data":576,"kb_used_omap":24,"kb_used_meta":26919,"kb_avail":104830064,"statfs":{"total":107374182400,"available":107345985536,"internally_reserved":0,"allocated":589824,"data_stored":270320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":25138,"internal_metadata":27565518},"hb_peers":[0,1,2,3,4,6,7,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":46,"seq":197568495632,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":27540,"kb_used_data":580,"kb_used_omap":25,"kb_used_meta":26918,"kb_avail":104830060,"statfs":{"total":107374182400,"available":107345981440,"internally_reserved":0,"allocated":593920,"data_stored":270339,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":26440,"internal_metadata":27564216},"hb_peers":[0,1,2,3,5,6,7,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":46,"seq":197568495632,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":27536,"kb_used_data":576,"kb_used_omap":22,"kb_used_meta":26921,"kb_avail":104830064,"statfs":{"total":107374182400,"available":107345985536,"internally_reserved":0,"allocated":589824,"data_stored":270320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":23183,"internal_metadata":27567473},"hb_peers":[0,1,2,4,5,6,7,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":46,"seq":197568495633,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":104857600,"kb_used":27560,"kb_used_data":600,"kb_used_omap":25,"kb_used_meta":26918,"kb_avail":104830040,"statfs":{"total":107374182400,"available":107345960960,"internally_reserved":0,"allocated":614400,"data_stored":288456,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":26440,"internal_metadata":27564216},"hb_peers":[0,1,3,4,5,6,7,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":46,"seq":197568495633,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":27552,"kb_used_data":592,"kb_used_omap":23,"kb_used_meta":26920,"kb_avail":104830048,"statfs":{"total":107374182400,"available":107345969152,"internally_reserved":0,"allocated":606208,"data_stored":280741,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":24487,"internal_metadata":27566169},"hb_peers":[0,2,3,4,5,6,7,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":46,"seq":197568495633,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":27536,"kb_used_data":576,"kb_used_omap":25,"kb_used_meta":26918,"kb_avail":104830064,"statfs":{"total":107374182400,"available":107345985536,"internally_reserved":0,"allocated":589824,"data_stored":270320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":25789,"internal_metadata":27564867},"hb_peers":[1,2,3,4,5,6,7,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T23:08:21.544 DEBUG:teuthology.orchestra.run.trial150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T23:08:21.719 INFO:teuthology.orchestra.run.trial150.stdout: 2026-02-21T23:08:21.720 INFO:teuthology.orchestra.run.trial150.stderr:dumped all 2026-02-21T23:08:21.727 INFO:teuthology.orchestra.run.trial150.stdout:{"pg_ready":true,"pg_map":{"version":96,"stamp":"2026-02-21T23:08:21.040084+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":10,"num_per_pool_osds":10,"num_per_pool_omap_osds":8,"kb":1048576000,"kb_used":275440,"kb_used_data":5840,"kb_used_omap":235,"kb_used_meta":269204,"kb_avail":1048300560,"statfs":{"total":1073741824000,"available":1073459773440,"internally_reserved":0,"allocated":5980160,"data_stored":2762982,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":241611,"internal_metadata":275664949},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.007211"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":62,"reported_epoch":52,"state":"active+clean","last_fresh":"2026-02-21T23:08:15.632899+0000","last_change":"2026-02-21T23:08:09.021397+0000","last_active":"2026-02-21T23:08:15.632899+0000","last_peered":"2026-02-21T23:08:15.632899+0000","last_clean":"2026-02-21T23:08:15.632899+0000","last_became_active":"2026-02-21T23:08:09.021232+0000","last_became_peered":"2026-02-21T23:08:09.021232+0000","last_unstale":"2026-02-21T23:08:15.632899+0000","last_undegraded":"2026-02-21T23:08:15.632899+0000","last_fullsized":"2026-02-21T23:08:15.632899+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T23:07:12.930820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T23:07:12.930820+0000","last_clean_scrub_stamp":"2026-02-21T23:07:12.930820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T01:01:37.847843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00033767299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":88,"reported_epoch":53,"state":"active+clean","last_fresh":"2026-02-21T23:08:15.632841+0000","last_change":"2026-02-21T23:08:09.021642+0000","last_active":"2026-02-21T23:08:15.632841+0000","last_peered":"2026-02-21T23:08:15.632841+0000","last_clean":"2026-02-21T23:08:15.632841+0000","last_became_active":"2026-02-21T23:08:09.021460+0000","last_became_peered":"2026-02-21T23:08:09.021460+0000","last_unstale":"2026-02-21T23:08:15.632841+0000","last_undegraded":"2026-02-21T23:08:15.632841+0000","last_fullsized":"2026-02-21T23:08:15.632841+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T23:07:12.930820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T23:07:12.930820+0000","last_clean_scrub_stamp":"2026-02-21T23:07:12.930820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T01:01:37.847843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022801700000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":88,"reported_epoch":53,"state":"active+clean","last_fresh":"2026-02-21T23:08:15.904902+0000","last_change":"2026-02-21T23:08:09.152799+0000","last_active":"2026-02-21T23:08:15.904902+0000","last_peered":"2026-02-21T23:08:15.904902+0000","last_clean":"2026-02-21T23:08:15.904902+0000","last_became_active":"2026-02-21T23:08:09.152693+0000","last_became_peered":"2026-02-21T23:08:09.152693+0000","last_unstale":"2026-02-21T23:08:15.904902+0000","last_undegraded":"2026-02-21T23:08:15.904902+0000","last_fullsized":"2026-02-21T23:08:15.904902+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T23:07:12.930820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T23:07:12.930820+0000","last_clean_scrub_stamp":"2026-02-21T23:07:12.930820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:57:13.040966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00029313000000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0],"acting":[9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":66,"reported_epoch":53,"state":"stale+active+clean","last_fresh":"2026-02-21T23:08:15.029819+0000","last_change":"2026-02-21T23:08:09.021356+0000","last_active":"2026-02-21T23:08:15.029819+0000","last_peered":"2026-02-21T23:08:15.029819+0000","last_clean":"2026-02-21T23:08:15.029819+0000","last_became_active":"2026-02-21T23:08:09.021241+0000","last_became_peered":"2026-02-21T23:08:09.021241+0000","last_unstale":"2026-02-21T23:08:18.033451+0000","last_undegraded":"2026-02-21T23:08:15.029819+0000","last_fullsized":"2026-02-21T23:08:15.029819+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T23:07:12.930820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T23:07:12.930820+0000","last_clean_scrub_stamp":"2026-02-21T23:07:12.930820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T01:42:39.532721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.3","version":"18'1","reported_seq":75,"reported_epoch":52,"state":"active+clean","last_fresh":"2026-02-21T23:08:15.632852+0000","last_change":"2026-02-21T23:08:09.021664+0000","last_active":"2026-02-21T23:08:15.632852+0000","last_peered":"2026-02-21T23:08:15.632852+0000","last_clean":"2026-02-21T23:08:15.632852+0000","last_became_active":"2026-02-21T23:08:09.021515+0000","last_became_peered":"2026-02-21T23:08:09.021515+0000","last_unstale":"2026-02-21T23:08:15.632852+0000","last_undegraded":"2026-02-21T23:08:15.632852+0000","last_fullsized":"2026-02-21T23:08:15.632852+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T23:07:12.930820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T23:07:12.930820+0000","last_clean_scrub_stamp":"2026-02-21T23:07:12.930820+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T02:56:28.927064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.2","version":"20'2","reported_seq":80,"reported_epoch":52,"state":"active+clean","last_fresh":"2026-02-21T23:08:15.632892+0000","last_change":"2026-02-21T23:08:09.021669+0000","last_active":"2026-02-21T23:08:15.632892+0000","last_peered":"2026-02-21T23:08:15.632892+0000","last_clean":"2026-02-21T23:08:15.632892+0000","last_became_active":"2026-02-21T23:08:09.021538+0000","last_became_peered":"2026-02-21T23:08:09.021538+0000","last_unstale":"2026-02-21T23:08:15.632892+0000","last_undegraded":"2026-02-21T23:08:15.632892+0000","last_fullsized":"2026-02-21T23:08:15.632892+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T23:07:12.930820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T23:07:12.930820+0000","last_clean_scrub_stamp":"2026-02-21T23:07:12.930820+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:12:38.733731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044592600000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":66,"reported_epoch":52,"state":"active+clean","last_fresh":"2026-02-21T23:08:15.632813+0000","last_change":"2026-02-21T23:08:09.021344+0000","last_active":"2026-02-21T23:08:15.632813+0000","last_peered":"2026-02-21T23:08:15.632813+0000","last_clean":"2026-02-21T23:08:15.632813+0000","last_became_active":"2026-02-21T23:08:09.021252+0000","last_became_peered":"2026-02-21T23:08:09.021252+0000","last_unstale":"2026-02-21T23:08:15.632813+0000","last_undegraded":"2026-02-21T23:08:15.632813+0000","last_fullsized":"2026-02-21T23:08:15.632813+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T23:07:12.930820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T23:07:12.930820+0000","last_clean_scrub_stamp":"2026-02-21T23:07:12.930820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:30:42.606649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00037064500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":74,"reported_epoch":52,"state":"active+clean","last_fresh":"2026-02-21T23:08:15.632841+0000","last_change":"2026-02-21T23:08:09.021203+0000","last_active":"2026-02-21T23:08:15.632841+0000","last_peered":"2026-02-21T23:08:15.632841+0000","last_clean":"2026-02-21T23:08:15.632841+0000","last_became_active":"2026-02-21T23:08:09.021076+0000","last_became_peered":"2026-02-21T23:08:09.021076+0000","last_unstale":"2026-02-21T23:08:15.632841+0000","last_undegraded":"2026-02-21T23:08:15.632841+0000","last_fullsized":"2026-02-21T23:08:15.632841+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T23:07:12.930820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T23:07:12.930820+0000","last_clean_scrub_stamp":"2026-02-21T23:07:12.930820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T00:21:40.020332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9],"acting":[5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":9}],"osd_stats":[{"osd":9,"up_from":40,"seq":171798691856,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":27548,"kb_used_data":588,"kb_used_omap":27,"kb_used_meta":26916,"kb_avail":104830052,"statfs":{"total":107374182400,"available":107345973248,"internally_reserved":0,"allocated":602112,"data_stored":280722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":27746,"internal_metadata":27562910},"hb_peers":[0,1,2,3,4,5,6,7,8],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":47,"seq":201863462929,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":104857600,"kb_used":27548,"kb_used_data":588,"kb_used_omap":23,"kb_used_meta":26920,"kb_avail":104830052,"statfs":{"total":107374182400,"available":107345973248,"internally_reserved":0,"allocated":602112,"data_stored":280722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":23836,"internal_metadata":27566820},"hb_peers":[0,1,2,3,4,5,6,7,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":46,"seq":197568495633,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":27536,"kb_used_data":576,"kb_used_omap":11,"kb_used_meta":26932,"kb_avail":104830064,"statfs":{"total":107374182400,"available":107345985536,"internally_reserved":0,"allocated":589824,"data_stored":270320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12113,"internal_metadata":27578543},"hb_peers":[0,1,2,3,4,5,6,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495632,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":27548,"kb_used_data":588,"kb_used_omap":25,"kb_used_meta":26918,"kb_avail":104830052,"statfs":{"total":107374182400,"available":107345973248,"internally_reserved":0,"allocated":602112,"data_stored":280722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":26439,"internal_metadata":27564217},"hb_peers":[0,1,2,3,4,5,7,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":46,"seq":197568495633,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":27536,"kb_used_data":576,"kb_used_omap":24,"kb_used_meta":26919,"kb_avail":104830064,"statfs":{"total":107374182400,"available":107345985536,"internally_reserved":0,"allocated":589824,"data_stored":270320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":25138,"internal_metadata":27565518},"hb_peers":[0,1,2,3,4,6,7,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":46,"seq":197568495632,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":27540,"kb_used_data":580,"kb_used_omap":25,"kb_used_meta":26918,"kb_avail":104830060,"statfs":{"total":107374182400,"available":107345981440,"internally_reserved":0,"allocated":593920,"data_stored":270339,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":26440,"internal_metadata":27564216},"hb_peers":[0,1,2,3,5,6,7,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":46,"seq":197568495632,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":27536,"kb_used_data":576,"kb_used_omap":22,"kb_used_meta":26921,"kb_avail":104830064,"statfs":{"total":107374182400,"available":107345985536,"internally_reserved":0,"allocated":589824,"data_stored":270320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":23183,"internal_metadata":27567473},"hb_peers":[0,1,2,4,5,6,7,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":46,"seq":197568495633,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":104857600,"kb_used":27560,"kb_used_data":600,"kb_used_omap":25,"kb_used_meta":26918,"kb_avail":104830040,"statfs":{"total":107374182400,"available":107345960960,"internally_reserved":0,"allocated":614400,"data_stored":288456,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":26440,"internal_metadata":27564216},"hb_peers":[0,1,3,4,5,6,7,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":46,"seq":197568495633,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":27552,"kb_used_data":592,"kb_used_omap":23,"kb_used_meta":26920,"kb_avail":104830048,"statfs":{"total":107374182400,"available":107345969152,"internally_reserved":0,"allocated":606208,"data_stored":280741,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":24487,"internal_metadata":27566169},"hb_peers":[0,2,3,4,5,6,7,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":46,"seq":197568495633,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":27536,"kb_used_data":576,"kb_used_omap":25,"kb_used_meta":26918,"kb_avail":104830064,"statfs":{"total":107374182400,"available":107345985536,"internally_reserved":0,"allocated":589824,"data_stored":270320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":25789,"internal_metadata":27564867},"hb_peers":[1,2,3,4,5,6,7,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T23:08:21.729 INFO:tasks.ceph.ceph_manager.ceph:PG 1.4 is not active+clean 2026-02-21T23:08:21.729 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '1.4', 'version': "0'0", 'reported_seq': 66, 'reported_epoch': 53, 'state': 'stale+active+clean', 'last_fresh': '2026-02-21T23:08:15.029819+0000', 'last_change': '2026-02-21T23:08:09.021356+0000', 'last_active': '2026-02-21T23:08:15.029819+0000', 'last_peered': '2026-02-21T23:08:15.029819+0000', 'last_clean': '2026-02-21T23:08:15.029819+0000', 'last_became_active': '2026-02-21T23:08:09.021241+0000', 'last_became_peered': '2026-02-21T23:08:09.021241+0000', 'last_unstale': '2026-02-21T23:08:18.033451+0000', 'last_undegraded': '2026-02-21T23:08:15.029819+0000', 'last_fullsized': '2026-02-21T23:08:15.029819+0000', 'mapping_epoch': 46, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 47, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2026-02-21T23:07:12.930820+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2026-02-21T23:07:12.930820+0000', 'last_clean_scrub_stamp': '2026-02-21T23:07:12.930820+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2026-02-23T01:42:39.532721+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2026-02-21T23:08:21.729 DEBUG:teuthology.orchestra.run.trial150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2026-02-21T23:08:21.951 INFO:teuthology.orchestra.run.trial150.stdout: 2026-02-21T23:08:21.952 INFO:teuthology.orchestra.run.trial150.stdout:{"fsid":"f77161f3-2466-452f-b0a1-014035b869f9","health":{"status":"HEALTH_WARN","checks":{"OSDMAP_FLAGS":{"severity":"HEALTH_WARN","summary":{"message":"noout flag(s) set","count":5},"muted":false}},"mutes":[]},"election_epoch":3,"quorum":[0],"quorum_names":["a"],"quorum_age":80,"monmap":{"epoch":1,"min_mon_release_name":"tentacle","num_mons":1},"osdmap":{"epoch":59,"num_osds":10,"num_up_osds":10,"osd_up_since":1771715300,"num_in_osds":10,"osd_in_since":1771715223,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":7},{"state_name":"stale+active+clean","count":1}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":282050560,"bytes_avail":1073459773440,"bytes_total":1073741824000},"fsmap":{"epoch":1,"btime":"2026-02-21T23:07:01:892305+0000","by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs"],"services":{}},"servicemap":{"epoch":3,"modified":"2026-02-21T23:08:08.820800+0000","services":{}},"progress_events":{}} 2026-02-21T23:08:21.961 INFO:tasks.ceph.ceph_manager.ceph:no progress seen, keeping timeout for now 2026-02-21T23:08:21.961 DEBUG:teuthology.orchestra.run.trial150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T23:08:22.139 INFO:teuthology.orchestra.run.trial150.stdout: 2026-02-21T23:08:22.140 INFO:teuthology.orchestra.run.trial150.stderr:dumped all 2026-02-21T23:08:22.148 INFO:teuthology.orchestra.run.trial150.stdout:{"pg_ready":true,"pg_map":{"version":96,"stamp":"2026-02-21T23:08:21.040084+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":10,"num_per_pool_osds":10,"num_per_pool_omap_osds":8,"kb":1048576000,"kb_used":275440,"kb_used_data":5840,"kb_used_omap":235,"kb_used_meta":269204,"kb_avail":1048300560,"statfs":{"total":1073741824000,"available":1073459773440,"internally_reserved":0,"allocated":5980160,"data_stored":2762982,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":241611,"internal_metadata":275664949},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.007211"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":62,"reported_epoch":52,"state":"active+clean","last_fresh":"2026-02-21T23:08:15.632899+0000","last_change":"2026-02-21T23:08:09.021397+0000","last_active":"2026-02-21T23:08:15.632899+0000","last_peered":"2026-02-21T23:08:15.632899+0000","last_clean":"2026-02-21T23:08:15.632899+0000","last_became_active":"2026-02-21T23:08:09.021232+0000","last_became_peered":"2026-02-21T23:08:09.021232+0000","last_unstale":"2026-02-21T23:08:15.632899+0000","last_undegraded":"2026-02-21T23:08:15.632899+0000","last_fullsized":"2026-02-21T23:08:15.632899+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T23:07:12.930820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T23:07:12.930820+0000","last_clean_scrub_stamp":"2026-02-21T23:07:12.930820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T01:01:37.847843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00033767299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":88,"reported_epoch":53,"state":"active+clean","last_fresh":"2026-02-21T23:08:15.632841+0000","last_change":"2026-02-21T23:08:09.021642+0000","last_active":"2026-02-21T23:08:15.632841+0000","last_peered":"2026-02-21T23:08:15.632841+0000","last_clean":"2026-02-21T23:08:15.632841+0000","last_became_active":"2026-02-21T23:08:09.021460+0000","last_became_peered":"2026-02-21T23:08:09.021460+0000","last_unstale":"2026-02-21T23:08:15.632841+0000","last_undegraded":"2026-02-21T23:08:15.632841+0000","last_fullsized":"2026-02-21T23:08:15.632841+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T23:07:12.930820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T23:07:12.930820+0000","last_clean_scrub_stamp":"2026-02-21T23:07:12.930820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T01:01:37.847843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022801700000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":88,"reported_epoch":53,"state":"active+clean","last_fresh":"2026-02-21T23:08:15.904902+0000","last_change":"2026-02-21T23:08:09.152799+0000","last_active":"2026-02-21T23:08:15.904902+0000","last_peered":"2026-02-21T23:08:15.904902+0000","last_clean":"2026-02-21T23:08:15.904902+0000","last_became_active":"2026-02-21T23:08:09.152693+0000","last_became_peered":"2026-02-21T23:08:09.152693+0000","last_unstale":"2026-02-21T23:08:15.904902+0000","last_undegraded":"2026-02-21T23:08:15.904902+0000","last_fullsized":"2026-02-21T23:08:15.904902+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T23:07:12.930820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T23:07:12.930820+0000","last_clean_scrub_stamp":"2026-02-21T23:07:12.930820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:57:13.040966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00029313000000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0],"acting":[9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":66,"reported_epoch":53,"state":"stale+active+clean","last_fresh":"2026-02-21T23:08:15.029819+0000","last_change":"2026-02-21T23:08:09.021356+0000","last_active":"2026-02-21T23:08:15.029819+0000","last_peered":"2026-02-21T23:08:15.029819+0000","last_clean":"2026-02-21T23:08:15.029819+0000","last_became_active":"2026-02-21T23:08:09.021241+0000","last_became_peered":"2026-02-21T23:08:09.021241+0000","last_unstale":"2026-02-21T23:08:18.033451+0000","last_undegraded":"2026-02-21T23:08:15.029819+0000","last_fullsized":"2026-02-21T23:08:15.029819+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T23:07:12.930820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T23:07:12.930820+0000","last_clean_scrub_stamp":"2026-02-21T23:07:12.930820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T01:42:39.532721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.3","version":"18'1","reported_seq":75,"reported_epoch":52,"state":"active+clean","last_fresh":"2026-02-21T23:08:15.632852+0000","last_change":"2026-02-21T23:08:09.021664+0000","last_active":"2026-02-21T23:08:15.632852+0000","last_peered":"2026-02-21T23:08:15.632852+0000","last_clean":"2026-02-21T23:08:15.632852+0000","last_became_active":"2026-02-21T23:08:09.021515+0000","last_became_peered":"2026-02-21T23:08:09.021515+0000","last_unstale":"2026-02-21T23:08:15.632852+0000","last_undegraded":"2026-02-21T23:08:15.632852+0000","last_fullsized":"2026-02-21T23:08:15.632852+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T23:07:12.930820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T23:07:12.930820+0000","last_clean_scrub_stamp":"2026-02-21T23:07:12.930820+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T02:56:28.927064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.2","version":"20'2","reported_seq":80,"reported_epoch":52,"state":"active+clean","last_fresh":"2026-02-21T23:08:15.632892+0000","last_change":"2026-02-21T23:08:09.021669+0000","last_active":"2026-02-21T23:08:15.632892+0000","last_peered":"2026-02-21T23:08:15.632892+0000","last_clean":"2026-02-21T23:08:15.632892+0000","last_became_active":"2026-02-21T23:08:09.021538+0000","last_became_peered":"2026-02-21T23:08:09.021538+0000","last_unstale":"2026-02-21T23:08:15.632892+0000","last_undegraded":"2026-02-21T23:08:15.632892+0000","last_fullsized":"2026-02-21T23:08:15.632892+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T23:07:12.930820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T23:07:12.930820+0000","last_clean_scrub_stamp":"2026-02-21T23:07:12.930820+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:12:38.733731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044592600000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":66,"reported_epoch":52,"state":"active+clean","last_fresh":"2026-02-21T23:08:15.632813+0000","last_change":"2026-02-21T23:08:09.021344+0000","last_active":"2026-02-21T23:08:15.632813+0000","last_peered":"2026-02-21T23:08:15.632813+0000","last_clean":"2026-02-21T23:08:15.632813+0000","last_became_active":"2026-02-21T23:08:09.021252+0000","last_became_peered":"2026-02-21T23:08:09.021252+0000","last_unstale":"2026-02-21T23:08:15.632813+0000","last_undegraded":"2026-02-21T23:08:15.632813+0000","last_fullsized":"2026-02-21T23:08:15.632813+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T23:07:12.930820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T23:07:12.930820+0000","last_clean_scrub_stamp":"2026-02-21T23:07:12.930820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:30:42.606649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00037064500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":74,"reported_epoch":52,"state":"active+clean","last_fresh":"2026-02-21T23:08:15.632841+0000","last_change":"2026-02-21T23:08:09.021203+0000","last_active":"2026-02-21T23:08:15.632841+0000","last_peered":"2026-02-21T23:08:15.632841+0000","last_clean":"2026-02-21T23:08:15.632841+0000","last_became_active":"2026-02-21T23:08:09.021076+0000","last_became_peered":"2026-02-21T23:08:09.021076+0000","last_unstale":"2026-02-21T23:08:15.632841+0000","last_undegraded":"2026-02-21T23:08:15.632841+0000","last_fullsized":"2026-02-21T23:08:15.632841+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T23:07:12.930820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T23:07:12.930820+0000","last_clean_scrub_stamp":"2026-02-21T23:07:12.930820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T00:21:40.020332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9],"acting":[5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":9}],"osd_stats":[{"osd":9,"up_from":40,"seq":171798691856,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":27548,"kb_used_data":588,"kb_used_omap":27,"kb_used_meta":26916,"kb_avail":104830052,"statfs":{"total":107374182400,"available":107345973248,"internally_reserved":0,"allocated":602112,"data_stored":280722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":27746,"internal_metadata":27562910},"hb_peers":[0,1,2,3,4,5,6,7,8],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":47,"seq":201863462929,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":104857600,"kb_used":27548,"kb_used_data":588,"kb_used_omap":23,"kb_used_meta":26920,"kb_avail":104830052,"statfs":{"total":107374182400,"available":107345973248,"internally_reserved":0,"allocated":602112,"data_stored":280722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":23836,"internal_metadata":27566820},"hb_peers":[0,1,2,3,4,5,6,7,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":46,"seq":197568495633,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":27536,"kb_used_data":576,"kb_used_omap":11,"kb_used_meta":26932,"kb_avail":104830064,"statfs":{"total":107374182400,"available":107345985536,"internally_reserved":0,"allocated":589824,"data_stored":270320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12113,"internal_metadata":27578543},"hb_peers":[0,1,2,3,4,5,6,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495632,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":27548,"kb_used_data":588,"kb_used_omap":25,"kb_used_meta":26918,"kb_avail":104830052,"statfs":{"total":107374182400,"available":107345973248,"internally_reserved":0,"allocated":602112,"data_stored":280722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":26439,"internal_metadata":27564217},"hb_peers":[0,1,2,3,4,5,7,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":46,"seq":197568495633,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":27536,"kb_used_data":576,"kb_used_omap":24,"kb_used_meta":26919,"kb_avail":104830064,"statfs":{"total":107374182400,"available":107345985536,"internally_reserved":0,"allocated":589824,"data_stored":270320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":25138,"internal_metadata":27565518},"hb_peers":[0,1,2,3,4,6,7,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":46,"seq":197568495632,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":27540,"kb_used_data":580,"kb_used_omap":25,"kb_used_meta":26918,"kb_avail":104830060,"statfs":{"total":107374182400,"available":107345981440,"internally_reserved":0,"allocated":593920,"data_stored":270339,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":26440,"internal_metadata":27564216},"hb_peers":[0,1,2,3,5,6,7,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":46,"seq":197568495632,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":27536,"kb_used_data":576,"kb_used_omap":22,"kb_used_meta":26921,"kb_avail":104830064,"statfs":{"total":107374182400,"available":107345985536,"internally_reserved":0,"allocated":589824,"data_stored":270320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":23183,"internal_metadata":27567473},"hb_peers":[0,1,2,4,5,6,7,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":46,"seq":197568495633,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":104857600,"kb_used":27560,"kb_used_data":600,"kb_used_omap":25,"kb_used_meta":26918,"kb_avail":104830040,"statfs":{"total":107374182400,"available":107345960960,"internally_reserved":0,"allocated":614400,"data_stored":288456,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":26440,"internal_metadata":27564216},"hb_peers":[0,1,3,4,5,6,7,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":46,"seq":197568495633,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":27552,"kb_used_data":592,"kb_used_omap":23,"kb_used_meta":26920,"kb_avail":104830048,"statfs":{"total":107374182400,"available":107345969152,"internally_reserved":0,"allocated":606208,"data_stored":280741,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":24487,"internal_metadata":27566169},"hb_peers":[0,2,3,4,5,6,7,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":46,"seq":197568495633,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":27536,"kb_used_data":576,"kb_used_omap":25,"kb_used_meta":26918,"kb_avail":104830064,"statfs":{"total":107374182400,"available":107345985536,"internally_reserved":0,"allocated":589824,"data_stored":270320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":25789,"internal_metadata":27564867},"hb_peers":[1,2,3,4,5,6,7,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T23:08:25.151 DEBUG:teuthology.orchestra.run.trial150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T23:08:25.327 INFO:teuthology.orchestra.run.trial150.stdout: 2026-02-21T23:08:25.328 INFO:teuthology.orchestra.run.trial150.stderr:dumped all 2026-02-21T23:08:25.336 INFO:teuthology.orchestra.run.trial150.stdout:{"pg_ready":true,"pg_map":{"version":98,"stamp":"2026-02-21T23:08:24.822586+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":10,"num_per_pool_osds":10,"num_per_pool_omap_osds":8,"kb":1048576000,"kb_used":276172,"kb_used_data":6572,"kb_used_omap":256,"kb_used_meta":269183,"kb_avail":1048299828,"statfs":{"total":1073741824000,"available":1073459023872,"internally_reserved":0,"allocated":6729728,"data_stored":3252364,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":263094,"internal_metadata":275643466},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.000723"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":73,"reported_epoch":59,"state":"active+clean","last_fresh":"2026-02-21T23:08:21.041254+0000","last_change":"2026-02-21T23:08:09.021397+0000","last_active":"2026-02-21T23:08:21.041254+0000","last_peered":"2026-02-21T23:08:21.041254+0000","last_clean":"2026-02-21T23:08:21.041254+0000","last_became_active":"2026-02-21T23:08:09.021232+0000","last_became_peered":"2026-02-21T23:08:09.021232+0000","last_unstale":"2026-02-21T23:08:21.041254+0000","last_undegraded":"2026-02-21T23:08:21.041254+0000","last_fullsized":"2026-02-21T23:08:21.041254+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T23:07:12.930820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T23:07:12.930820+0000","last_clean_scrub_stamp":"2026-02-21T23:07:12.930820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T01:01:37.847843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00033767299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":98,"reported_epoch":59,"state":"active+clean","last_fresh":"2026-02-21T23:08:21.040673+0000","last_change":"2026-02-21T23:08:09.021642+0000","last_active":"2026-02-21T23:08:21.040673+0000","last_peered":"2026-02-21T23:08:21.040673+0000","last_clean":"2026-02-21T23:08:21.040673+0000","last_became_active":"2026-02-21T23:08:09.021460+0000","last_became_peered":"2026-02-21T23:08:09.021460+0000","last_unstale":"2026-02-21T23:08:21.040673+0000","last_undegraded":"2026-02-21T23:08:21.040673+0000","last_fullsized":"2026-02-21T23:08:21.040673+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T23:07:12.930820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T23:07:12.930820+0000","last_clean_scrub_stamp":"2026-02-21T23:07:12.930820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T01:01:37.847843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022801700000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":107,"reported_epoch":59,"state":"active+clean","last_fresh":"2026-02-21T23:08:21.633705+0000","last_change":"2026-02-21T23:08:20.150609+0000","last_active":"2026-02-21T23:08:21.633705+0000","last_peered":"2026-02-21T23:08:21.633705+0000","last_clean":"2026-02-21T23:08:21.633705+0000","last_became_active":"2026-02-21T23:08:20.150454+0000","last_became_peered":"2026-02-21T23:08:20.150454+0000","last_unstale":"2026-02-21T23:08:21.633705+0000","last_undegraded":"2026-02-21T23:08:21.633705+0000","last_fullsized":"2026-02-21T23:08:21.633705+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T23:07:12.930820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T23:07:12.930820+0000","last_clean_scrub_stamp":"2026-02-21T23:07:12.930820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T02:18:46.454644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00029313000000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0],"acting":[9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":78,"reported_epoch":59,"state":"active+clean","last_fresh":"2026-02-21T23:08:21.041583+0000","last_change":"2026-02-21T23:08:21.041523+0000","last_active":"2026-02-21T23:08:21.041583+0000","last_peered":"2026-02-21T23:08:21.041583+0000","last_clean":"2026-02-21T23:08:21.041583+0000","last_became_active":"2026-02-21T23:08:21.041355+0000","last_became_peered":"2026-02-21T23:08:21.041355+0000","last_unstale":"2026-02-21T23:08:21.041583+0000","last_undegraded":"2026-02-21T23:08:21.041583+0000","last_fullsized":"2026-02-21T23:08:21.041583+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T23:07:12.930820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T23:07:12.930820+0000","last_clean_scrub_stamp":"2026-02-21T23:07:12.930820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T07:10:36.306892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.3","version":"18'1","reported_seq":94,"reported_epoch":59,"state":"active+clean","last_fresh":"2026-02-21T23:08:21.041976+0000","last_change":"2026-02-21T23:08:21.041938+0000","last_active":"2026-02-21T23:08:21.041976+0000","last_peered":"2026-02-21T23:08:21.041976+0000","last_clean":"2026-02-21T23:08:21.041976+0000","last_became_active":"2026-02-21T23:08:21.041805+0000","last_became_peered":"2026-02-21T23:08:21.041805+0000","last_unstale":"2026-02-21T23:08:21.041976+0000","last_undegraded":"2026-02-21T23:08:21.041976+0000","last_fullsized":"2026-02-21T23:08:21.041976+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T23:07:12.930820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T23:07:12.930820+0000","last_clean_scrub_stamp":"2026-02-21T23:07:12.930820+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:30:21.881362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.2","version":"20'2","reported_seq":103,"reported_epoch":59,"state":"active+clean","last_fresh":"2026-02-21T23:08:21.041702+0000","last_change":"2026-02-21T23:08:21.041641+0000","last_active":"2026-02-21T23:08:21.041702+0000","last_peered":"2026-02-21T23:08:21.041702+0000","last_clean":"2026-02-21T23:08:21.041702+0000","last_became_active":"2026-02-21T23:08:21.041420+0000","last_became_peered":"2026-02-21T23:08:21.041420+0000","last_unstale":"2026-02-21T23:08:21.041702+0000","last_undegraded":"2026-02-21T23:08:21.041702+0000","last_fullsized":"2026-02-21T23:08:21.041702+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T23:07:12.930820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T23:07:12.930820+0000","last_clean_scrub_stamp":"2026-02-21T23:07:12.930820+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:14:00.344365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044592600000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":77,"reported_epoch":59,"state":"active+clean","last_fresh":"2026-02-21T23:08:21.040718+0000","last_change":"2026-02-21T23:08:09.021344+0000","last_active":"2026-02-21T23:08:21.040718+0000","last_peered":"2026-02-21T23:08:21.040718+0000","last_clean":"2026-02-21T23:08:21.040718+0000","last_became_active":"2026-02-21T23:08:09.021252+0000","last_became_peered":"2026-02-21T23:08:09.021252+0000","last_unstale":"2026-02-21T23:08:21.040718+0000","last_undegraded":"2026-02-21T23:08:21.040718+0000","last_fullsized":"2026-02-21T23:08:21.040718+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T23:07:12.930820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T23:07:12.930820+0000","last_clean_scrub_stamp":"2026-02-21T23:07:12.930820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:30:42.606649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00037064500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":85,"reported_epoch":59,"state":"active+clean","last_fresh":"2026-02-21T23:08:21.040761+0000","last_change":"2026-02-21T23:08:09.021203+0000","last_active":"2026-02-21T23:08:21.040761+0000","last_peered":"2026-02-21T23:08:21.040761+0000","last_clean":"2026-02-21T23:08:21.040761+0000","last_became_active":"2026-02-21T23:08:09.021076+0000","last_became_peered":"2026-02-21T23:08:09.021076+0000","last_unstale":"2026-02-21T23:08:21.040761+0000","last_undegraded":"2026-02-21T23:08:21.040761+0000","last_fullsized":"2026-02-21T23:08:21.040761+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T23:07:12.930820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T23:07:12.930820+0000","last_clean_scrub_stamp":"2026-02-21T23:07:12.930820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T00:21:40.020332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9],"acting":[5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":9}],"osd_stats":[{"osd":9,"up_from":40,"seq":171798691857,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":27620,"kb_used_data":660,"kb_used_omap":30,"kb_used_meta":26913,"kb_avail":104829980,"statfs":{"total":107374182400,"available":107345899520,"internally_reserved":0,"allocated":675840,"data_stored":327530,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":31001,"internal_metadata":27559655},"hb_peers":[0,1,2,3,4,5,6,7,8],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":47,"seq":201863462930,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":104857600,"kb_used":27608,"kb_used_data":648,"kb_used_omap":25,"kb_used_meta":26918,"kb_avail":104829992,"statfs":{"total":107374182400,"available":107345911808,"internally_reserved":0,"allocated":663552,"data_stored":319872,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":25789,"internal_metadata":27564867},"hb_peers":[2,3,4,5,6,7,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":46,"seq":197568495634,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":27608,"kb_used_data":648,"kb_used_omap":11,"kb_used_meta":26932,"kb_avail":104829992,"statfs":{"total":107374182400,"available":107345911808,"internally_reserved":0,"allocated":663552,"data_stored":319872,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12113,"internal_metadata":27578543},"hb_peers":[0,1,2,3,4,5,6,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495633,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":27620,"kb_used_data":660,"kb_used_omap":28,"kb_used_meta":26915,"kb_avail":104829980,"statfs":{"total":107374182400,"available":107345899520,"internally_reserved":0,"allocated":675840,"data_stored":327530,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":29043,"internal_metadata":27561613},"hb_peers":[2,3,4,5,7,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":46,"seq":197568495634,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":27620,"kb_used_data":660,"kb_used_omap":27,"kb_used_meta":26916,"kb_avail":104829980,"statfs":{"total":107374182400,"available":107345899520,"internally_reserved":0,"allocated":675840,"data_stored":327530,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":27742,"internal_metadata":27562914},"hb_peers":[0,1,2,3,4,6,7,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":46,"seq":197568495633,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":27612,"kb_used_data":652,"kb_used_omap":29,"kb_used_meta":26914,"kb_avail":104829988,"statfs":{"total":107374182400,"available":107345907712,"internally_reserved":0,"allocated":667648,"data_stored":319891,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":30346,"internal_metadata":27560310},"hb_peers":[0,1,2,3,5,6,7,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":46,"seq":197568495633,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":27620,"kb_used_data":660,"kb_used_omap":22,"kb_used_meta":26921,"kb_avail":104829980,"statfs":{"total":107374182400,"available":107345899520,"internally_reserved":0,"allocated":675840,"data_stored":327530,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":23183,"internal_metadata":27567473},"hb_peers":[2,4,5,6,7,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":46,"seq":197568495634,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":104857600,"kb_used":27620,"kb_used_data":660,"kb_used_omap":28,"kb_used_meta":26915,"kb_avail":104829980,"statfs":{"total":107374182400,"available":107345899520,"internally_reserved":0,"allocated":675840,"data_stored":327530,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":29044,"internal_metadata":27561612},"hb_peers":[0,1,3,4,5,6,7,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":58,"seq":249108103186,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":27624,"kb_used_data":664,"kb_used_omap":25,"kb_used_meta":26918,"kb_avail":104829976,"statfs":{"total":107374182400,"available":107345895424,"internally_reserved":0,"allocated":679936,"data_stored":327549,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":25789,"internal_metadata":27564867},"hb_peers":[0,2,3,4,5,6,7,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":57,"seq":244813135890,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":27620,"kb_used_data":660,"kb_used_omap":28,"kb_used_meta":26915,"kb_avail":104829980,"statfs":{"total":107374182400,"available":107345899520,"internally_reserved":0,"allocated":675840,"data_stored":327530,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":29044,"internal_metadata":27561612},"hb_peers":[2,3,4,5,6,7,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T23:08:25.337 INFO:tasks.ceph.ceph_manager.ceph:clean! 2026-02-21T23:08:25.337 DEBUG:teuthology.orchestra.run.trial150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T23:08:25.516 INFO:teuthology.orchestra.run.trial150.stdout: 2026-02-21T23:08:25.516 INFO:teuthology.orchestra.run.trial150.stderr:dumped all 2026-02-21T23:08:25.525 INFO:teuthology.orchestra.run.trial150.stdout:{"pg_ready":true,"pg_map":{"version":98,"stamp":"2026-02-21T23:08:24.822586+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":10,"num_per_pool_osds":10,"num_per_pool_omap_osds":8,"kb":1048576000,"kb_used":276172,"kb_used_data":6572,"kb_used_omap":256,"kb_used_meta":269183,"kb_avail":1048299828,"statfs":{"total":1073741824000,"available":1073459023872,"internally_reserved":0,"allocated":6729728,"data_stored":3252364,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":263094,"internal_metadata":275643466},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.000723"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":73,"reported_epoch":59,"state":"active+clean","last_fresh":"2026-02-21T23:08:21.041254+0000","last_change":"2026-02-21T23:08:09.021397+0000","last_active":"2026-02-21T23:08:21.041254+0000","last_peered":"2026-02-21T23:08:21.041254+0000","last_clean":"2026-02-21T23:08:21.041254+0000","last_became_active":"2026-02-21T23:08:09.021232+0000","last_became_peered":"2026-02-21T23:08:09.021232+0000","last_unstale":"2026-02-21T23:08:21.041254+0000","last_undegraded":"2026-02-21T23:08:21.041254+0000","last_fullsized":"2026-02-21T23:08:21.041254+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T23:07:12.930820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T23:07:12.930820+0000","last_clean_scrub_stamp":"2026-02-21T23:07:12.930820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T01:01:37.847843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00033767299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":98,"reported_epoch":59,"state":"active+clean","last_fresh":"2026-02-21T23:08:21.040673+0000","last_change":"2026-02-21T23:08:09.021642+0000","last_active":"2026-02-21T23:08:21.040673+0000","last_peered":"2026-02-21T23:08:21.040673+0000","last_clean":"2026-02-21T23:08:21.040673+0000","last_became_active":"2026-02-21T23:08:09.021460+0000","last_became_peered":"2026-02-21T23:08:09.021460+0000","last_unstale":"2026-02-21T23:08:21.040673+0000","last_undegraded":"2026-02-21T23:08:21.040673+0000","last_fullsized":"2026-02-21T23:08:21.040673+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T23:07:12.930820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T23:07:12.930820+0000","last_clean_scrub_stamp":"2026-02-21T23:07:12.930820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T01:01:37.847843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022801700000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":107,"reported_epoch":59,"state":"active+clean","last_fresh":"2026-02-21T23:08:21.633705+0000","last_change":"2026-02-21T23:08:20.150609+0000","last_active":"2026-02-21T23:08:21.633705+0000","last_peered":"2026-02-21T23:08:21.633705+0000","last_clean":"2026-02-21T23:08:21.633705+0000","last_became_active":"2026-02-21T23:08:20.150454+0000","last_became_peered":"2026-02-21T23:08:20.150454+0000","last_unstale":"2026-02-21T23:08:21.633705+0000","last_undegraded":"2026-02-21T23:08:21.633705+0000","last_fullsized":"2026-02-21T23:08:21.633705+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T23:07:12.930820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T23:07:12.930820+0000","last_clean_scrub_stamp":"2026-02-21T23:07:12.930820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T02:18:46.454644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00029313000000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0],"acting":[9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":78,"reported_epoch":59,"state":"active+clean","last_fresh":"2026-02-21T23:08:21.041583+0000","last_change":"2026-02-21T23:08:21.041523+0000","last_active":"2026-02-21T23:08:21.041583+0000","last_peered":"2026-02-21T23:08:21.041583+0000","last_clean":"2026-02-21T23:08:21.041583+0000","last_became_active":"2026-02-21T23:08:21.041355+0000","last_became_peered":"2026-02-21T23:08:21.041355+0000","last_unstale":"2026-02-21T23:08:21.041583+0000","last_undegraded":"2026-02-21T23:08:21.041583+0000","last_fullsized":"2026-02-21T23:08:21.041583+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T23:07:12.930820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T23:07:12.930820+0000","last_clean_scrub_stamp":"2026-02-21T23:07:12.930820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T07:10:36.306892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.3","version":"18'1","reported_seq":94,"reported_epoch":59,"state":"active+clean","last_fresh":"2026-02-21T23:08:21.041976+0000","last_change":"2026-02-21T23:08:21.041938+0000","last_active":"2026-02-21T23:08:21.041976+0000","last_peered":"2026-02-21T23:08:21.041976+0000","last_clean":"2026-02-21T23:08:21.041976+0000","last_became_active":"2026-02-21T23:08:21.041805+0000","last_became_peered":"2026-02-21T23:08:21.041805+0000","last_unstale":"2026-02-21T23:08:21.041976+0000","last_undegraded":"2026-02-21T23:08:21.041976+0000","last_fullsized":"2026-02-21T23:08:21.041976+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T23:07:12.930820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T23:07:12.930820+0000","last_clean_scrub_stamp":"2026-02-21T23:07:12.930820+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:30:21.881362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.2","version":"20'2","reported_seq":103,"reported_epoch":59,"state":"active+clean","last_fresh":"2026-02-21T23:08:21.041702+0000","last_change":"2026-02-21T23:08:21.041641+0000","last_active":"2026-02-21T23:08:21.041702+0000","last_peered":"2026-02-21T23:08:21.041702+0000","last_clean":"2026-02-21T23:08:21.041702+0000","last_became_active":"2026-02-21T23:08:21.041420+0000","last_became_peered":"2026-02-21T23:08:21.041420+0000","last_unstale":"2026-02-21T23:08:21.041702+0000","last_undegraded":"2026-02-21T23:08:21.041702+0000","last_fullsized":"2026-02-21T23:08:21.041702+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T23:07:12.930820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T23:07:12.930820+0000","last_clean_scrub_stamp":"2026-02-21T23:07:12.930820+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:14:00.344365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044592600000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":77,"reported_epoch":59,"state":"active+clean","last_fresh":"2026-02-21T23:08:21.040718+0000","last_change":"2026-02-21T23:08:09.021344+0000","last_active":"2026-02-21T23:08:21.040718+0000","last_peered":"2026-02-21T23:08:21.040718+0000","last_clean":"2026-02-21T23:08:21.040718+0000","last_became_active":"2026-02-21T23:08:09.021252+0000","last_became_peered":"2026-02-21T23:08:09.021252+0000","last_unstale":"2026-02-21T23:08:21.040718+0000","last_undegraded":"2026-02-21T23:08:21.040718+0000","last_fullsized":"2026-02-21T23:08:21.040718+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T23:07:12.930820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T23:07:12.930820+0000","last_clean_scrub_stamp":"2026-02-21T23:07:12.930820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:30:42.606649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00037064500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":85,"reported_epoch":59,"state":"active+clean","last_fresh":"2026-02-21T23:08:21.040761+0000","last_change":"2026-02-21T23:08:09.021203+0000","last_active":"2026-02-21T23:08:21.040761+0000","last_peered":"2026-02-21T23:08:21.040761+0000","last_clean":"2026-02-21T23:08:21.040761+0000","last_became_active":"2026-02-21T23:08:09.021076+0000","last_became_peered":"2026-02-21T23:08:09.021076+0000","last_unstale":"2026-02-21T23:08:21.040761+0000","last_undegraded":"2026-02-21T23:08:21.040761+0000","last_fullsized":"2026-02-21T23:08:21.040761+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T23:07:12.930820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T23:07:12.930820+0000","last_clean_scrub_stamp":"2026-02-21T23:07:12.930820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T00:21:40.020332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9],"acting":[5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":9}],"osd_stats":[{"osd":9,"up_from":40,"seq":171798691857,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":27620,"kb_used_data":660,"kb_used_omap":30,"kb_used_meta":26913,"kb_avail":104829980,"statfs":{"total":107374182400,"available":107345899520,"internally_reserved":0,"allocated":675840,"data_stored":327530,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":31001,"internal_metadata":27559655},"hb_peers":[0,1,2,3,4,5,6,7,8],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":47,"seq":201863462930,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":104857600,"kb_used":27608,"kb_used_data":648,"kb_used_omap":25,"kb_used_meta":26918,"kb_avail":104829992,"statfs":{"total":107374182400,"available":107345911808,"internally_reserved":0,"allocated":663552,"data_stored":319872,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":25789,"internal_metadata":27564867},"hb_peers":[2,3,4,5,6,7,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":46,"seq":197568495634,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":27608,"kb_used_data":648,"kb_used_omap":11,"kb_used_meta":26932,"kb_avail":104829992,"statfs":{"total":107374182400,"available":107345911808,"internally_reserved":0,"allocated":663552,"data_stored":319872,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12113,"internal_metadata":27578543},"hb_peers":[0,1,2,3,4,5,6,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495633,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":27620,"kb_used_data":660,"kb_used_omap":28,"kb_used_meta":26915,"kb_avail":104829980,"statfs":{"total":107374182400,"available":107345899520,"internally_reserved":0,"allocated":675840,"data_stored":327530,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":29043,"internal_metadata":27561613},"hb_peers":[2,3,4,5,7,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":46,"seq":197568495634,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":27620,"kb_used_data":660,"kb_used_omap":27,"kb_used_meta":26916,"kb_avail":104829980,"statfs":{"total":107374182400,"available":107345899520,"internally_reserved":0,"allocated":675840,"data_stored":327530,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":27742,"internal_metadata":27562914},"hb_peers":[0,1,2,3,4,6,7,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":46,"seq":197568495633,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":27612,"kb_used_data":652,"kb_used_omap":29,"kb_used_meta":26914,"kb_avail":104829988,"statfs":{"total":107374182400,"available":107345907712,"internally_reserved":0,"allocated":667648,"data_stored":319891,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":30346,"internal_metadata":27560310},"hb_peers":[0,1,2,3,5,6,7,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":46,"seq":197568495633,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":27620,"kb_used_data":660,"kb_used_omap":22,"kb_used_meta":26921,"kb_avail":104829980,"statfs":{"total":107374182400,"available":107345899520,"internally_reserved":0,"allocated":675840,"data_stored":327530,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":23183,"internal_metadata":27567473},"hb_peers":[2,4,5,6,7,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":46,"seq":197568495634,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":104857600,"kb_used":27620,"kb_used_data":660,"kb_used_omap":28,"kb_used_meta":26915,"kb_avail":104829980,"statfs":{"total":107374182400,"available":107345899520,"internally_reserved":0,"allocated":675840,"data_stored":327530,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":29044,"internal_metadata":27561612},"hb_peers":[0,1,3,4,5,6,7,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":58,"seq":249108103186,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":27624,"kb_used_data":664,"kb_used_omap":25,"kb_used_meta":26918,"kb_avail":104829976,"statfs":{"total":107374182400,"available":107345895424,"internally_reserved":0,"allocated":679936,"data_stored":327549,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":25789,"internal_metadata":27564867},"hb_peers":[0,2,3,4,5,6,7,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":57,"seq":244813135890,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":27620,"kb_used_data":660,"kb_used_omap":28,"kb_used_meta":26915,"kb_avail":104829980,"statfs":{"total":107374182400,"available":107345899520,"internally_reserved":0,"allocated":675840,"data_stored":327530,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":29044,"internal_metadata":27561612},"hb_peers":[2,3,4,5,6,7,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T23:08:25.527 DEBUG:teuthology.orchestra.run.trial150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2026-02-21T23:08:25.704 INFO:teuthology.orchestra.run.trial150.stdout: 2026-02-21T23:08:25.704 INFO:teuthology.orchestra.run.trial150.stdout:{"epoch":59,"fsid":"f77161f3-2466-452f-b0a1-014035b869f9","created":"2026-02-21T23:07:01.892470+0000","modified":"2026-02-21T23:08:21.039057+0000","last_up_change":"2026-02-21T23:08:20.036619+0000","last_in_change":"2026-02-21T23:07:03.803679+0000","flags":"noout,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5800064,"flags_set":["noout","pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":10,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":10,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"tentacle","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2026-02-21T23:07:12.576318+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":20,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"nonprimary_shards":"{}","options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":0.80000001192092896,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1.0000001192092896,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c4ec8e56-7376-4e6a-84af-d03afaa3dbee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":15,"last_clean_end":56,"up_from":57,"up_thru":36,"down_at":54,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.150:6858","nonce":1860146791},{"type":"v1","addr":"10.20.193.150:6860","nonce":1860146791}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.150:6841","nonce":1863146791},{"type":"v1","addr":"10.20.193.150:6846","nonce":1863146791}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.150:6870","nonce":1860146791},{"type":"v1","addr":"10.20.193.150:6872","nonce":1860146791}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.150:6866","nonce":1860146791},{"type":"v1","addr":"10.20.193.150:6868","nonce":1860146791}]},"public_addr":"10.20.193.150:6860/1860146791","cluster_addr":"10.20.193.150:6846/1863146791","heartbeat_back_addr":"10.20.193.150:6872/1860146791","heartbeat_front_addr":"10.20.193.150:6868/1860146791","state":["exists","up"]},{"osd":1,"uuid":"389ddd9e-6beb-48ce-8ee1-574db38a866c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":15,"last_clean_end":57,"up_from":58,"up_thru":58,"down_at":54,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.150:6810","nonce":2256230450},{"type":"v1","addr":"10.20.193.150:6811","nonce":2256230450}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.150:6863","nonce":2259230450},{"type":"v1","addr":"10.20.193.150:6865","nonce":2259230450}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.150:6816","nonce":2256230450},{"type":"v1","addr":"10.20.193.150:6817","nonce":2256230450}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.150:6814","nonce":2256230450},{"type":"v1","addr":"10.20.193.150:6815","nonce":2256230450}]},"public_addr":"10.20.193.150:6811/2256230450","cluster_addr":"10.20.193.150:6865/2259230450","heartbeat_back_addr":"10.20.193.150:6817/2256230450","heartbeat_front_addr":"10.20.193.150:6815/2256230450","state":["exists","up"]},{"osd":2,"uuid":"718e49c5-1294-46b1-b1b4-fa3edda8564c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":15,"last_clean_end":45,"up_from":46,"up_thru":0,"down_at":43,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.150:6826","nonce":3930449501},{"type":"v1","addr":"10.20.193.150:6828","nonce":3930449501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.150:6804","nonce":3931449501},{"type":"v1","addr":"10.20.193.150:6805","nonce":3931449501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.150:6844","nonce":3930449501},{"type":"v1","addr":"10.20.193.150:6847","nonce":3930449501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.150:6837","nonce":3930449501},{"type":"v1","addr":"10.20.193.150:6840","nonce":3930449501}]},"public_addr":"10.20.193.150:6828/3930449501","cluster_addr":"10.20.193.150:6805/3931449501","heartbeat_back_addr":"10.20.193.150:6847/3930449501","heartbeat_front_addr":"10.20.193.150:6840/3930449501","state":["exists","up"]},{"osd":3,"uuid":"9a09d418-342f-4db0-a6e1-700059b6f0f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":15,"last_clean_end":45,"up_from":46,"up_thru":58,"down_at":43,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.150:6834","nonce":317686896},{"type":"v1","addr":"10.20.193.150:6839","nonce":317686896}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.150:6820","nonce":318686896},{"type":"v1","addr":"10.20.193.150:6821","nonce":318686896}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.150:6855","nonce":317686896},{"type":"v1","addr":"10.20.193.150:6857","nonce":317686896}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.150:6851","nonce":317686896},{"type":"v1","addr":"10.20.193.150:6853","nonce":317686896}]},"public_addr":"10.20.193.150:6839/317686896","cluster_addr":"10.20.193.150:6821/318686896","heartbeat_back_addr":"10.20.193.150:6857/317686896","heartbeat_front_addr":"10.20.193.150:6853/317686896","state":["exists","up"]},{"osd":4,"uuid":"e19f379d-43d8-4536-ac93-ad0d30c98828","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":15,"last_clean_end":45,"up_from":46,"up_thru":58,"down_at":43,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.150:6802","nonce":1337979806},{"type":"v1","addr":"10.20.193.150:6803","nonce":1337979806}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.150:6830","nonce":1338979806},{"type":"v1","addr":"10.20.193.150:6831","nonce":1338979806}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.150:6808","nonce":1337979806},{"type":"v1","addr":"10.20.193.150:6809","nonce":1337979806}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.150:6806","nonce":1337979806},{"type":"v1","addr":"10.20.193.150:6807","nonce":1337979806}]},"public_addr":"10.20.193.150:6803/1337979806","cluster_addr":"10.20.193.150:6831/1338979806","heartbeat_back_addr":"10.20.193.150:6809/1337979806","heartbeat_front_addr":"10.20.193.150:6807/1337979806","state":["exists","up"]},{"osd":5,"uuid":"5d7c9240-c8e9-40d3-835b-ded2969b7c24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":15,"last_clean_end":45,"up_from":46,"up_thru":46,"down_at":43,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.150:6874","nonce":1455759585},{"type":"v1","addr":"10.20.193.150:6875","nonce":1455759585}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.150:6835","nonce":1456759585},{"type":"v1","addr":"10.20.193.150:6848","nonce":1456759585}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.150:6880","nonce":1455759585},{"type":"v1","addr":"10.20.193.150:6881","nonce":1455759585}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.150:6878","nonce":1455759585},{"type":"v1","addr":"10.20.193.150:6879","nonce":1455759585}]},"public_addr":"10.20.193.150:6875/1455759585","cluster_addr":"10.20.193.150:6848/1456759585","heartbeat_back_addr":"10.20.193.150:6881/1455759585","heartbeat_front_addr":"10.20.193.150:6879/1455759585","state":["exists","up"]},{"osd":6,"uuid":"0978d144-b749-4b5e-9c84-9a6c2f1bc3dc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":15,"last_clean_end":45,"up_from":46,"up_thru":55,"down_at":43,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.150:6818","nonce":1341103058},{"type":"v1","addr":"10.20.193.150:6819","nonce":1341103058}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.150:6862","nonce":1342103058},{"type":"v1","addr":"10.20.193.150:6864","nonce":1342103058}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.150:6824","nonce":1341103058},{"type":"v1","addr":"10.20.193.150:6825","nonce":1341103058}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.150:6822","nonce":1341103058},{"type":"v1","addr":"10.20.193.150:6823","nonce":1341103058}]},"public_addr":"10.20.193.150:6819/1341103058","cluster_addr":"10.20.193.150:6864/1342103058","heartbeat_back_addr":"10.20.193.150:6825/1341103058","heartbeat_front_addr":"10.20.193.150:6823/1341103058","state":["exists","up"]},{"osd":7,"uuid":"06c1e800-e425-4898-9bbf-fbf1b196d33e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":15,"last_clean_end":45,"up_from":46,"up_thru":46,"down_at":43,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.150:6827","nonce":1453052406},{"type":"v1","addr":"10.20.193.150:6829","nonce":1453052406}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.150:6833","nonce":1454052406},{"type":"v1","addr":"10.20.193.150:6843","nonce":1454052406}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.150:6845","nonce":1453052406},{"type":"v1","addr":"10.20.193.150:6849","nonce":1453052406}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.150:6838","nonce":1453052406},{"type":"v1","addr":"10.20.193.150:6842","nonce":1453052406}]},"public_addr":"10.20.193.150:6829/1453052406","cluster_addr":"10.20.193.150:6843/1454052406","heartbeat_back_addr":"10.20.193.150:6849/1453052406","heartbeat_front_addr":"10.20.193.150:6842/1453052406","state":["exists","up"]},{"osd":8,"uuid":"52f40cc0-0808-4b7d-a711-e77d7d19ff58","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":15,"last_clean_end":46,"up_from":47,"up_thru":17,"down_at":43,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.150:6859","nonce":2701237949},{"type":"v1","addr":"10.20.193.150:6861","nonce":2701237949}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.150:6876","nonce":2703237949},{"type":"v1","addr":"10.20.193.150:6877","nonce":2703237949}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.150:6871","nonce":2701237949},{"type":"v1","addr":"10.20.193.150:6873","nonce":2701237949}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.150:6867","nonce":2701237949},{"type":"v1","addr":"10.20.193.150:6869","nonce":2701237949}]},"public_addr":"10.20.193.150:6861/2701237949","cluster_addr":"10.20.193.150:6877/2703237949","heartbeat_back_addr":"10.20.193.150:6873/2701237949","heartbeat_front_addr":"10.20.193.150:6869/2701237949","state":["exists","up"]},{"osd":9,"uuid":"74e2070c-75ed-4e1b-b25a-86614f45a1fb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":15,"last_clean_end":39,"up_from":40,"up_thru":57,"down_at":36,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.150:6832","nonce":2439465072},{"type":"v1","addr":"10.20.193.150:6836","nonce":2439465072}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.150:6812","nonce":2440465072},{"type":"v1","addr":"10.20.193.150:6813","nonce":2440465072}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.150:6854","nonce":2439465072},{"type":"v1","addr":"10.20.193.150:6856","nonce":2439465072}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.150:6850","nonce":2439465072},{"type":"v1","addr":"10.20.193.150:6852","nonce":2439465072}]},"public_addr":"10.20.193.150:6836/2439465072","cluster_addr":"10.20.193.150:6813/2440465072","heartbeat_back_addr":"10.20.193.150:6856/2439465072","heartbeat_front_addr":"10.20.193.150:6852/2439465072","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"2026-02-21T23:08:16.029232+0000","laggy_probability":0.65700000524520874,"laggy_interval":1,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-21T23:07:05.499158+0000","dead_epoch":56},{"osd":1,"down_stamp":"2026-02-21T23:08:16.029232+0000","laggy_probability":0.65700000524520874,"laggy_interval":1,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-21T23:07:05.477193+0000","dead_epoch":57},{"osd":2,"down_stamp":"2026-02-21T23:08:05.005884+0000","laggy_probability":0.30000001192092896,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-21T23:07:05.479491+0000","dead_epoch":43},{"osd":3,"down_stamp":"2026-02-21T23:08:05.005884+0000","laggy_probability":0.30000001192092896,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-21T23:07:05.482361+0000","dead_epoch":43},{"osd":4,"down_stamp":"2026-02-21T23:08:05.005884+0000","laggy_probability":0.30000001192092896,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-21T23:07:05.561036+0000","dead_epoch":43},{"osd":5,"down_stamp":"2026-02-21T23:08:05.005884+0000","laggy_probability":0.30000001192092896,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-21T23:07:05.555880+0000","dead_epoch":43},{"osd":6,"down_stamp":"2026-02-21T23:08:05.005884+0000","laggy_probability":0.30000001192092896,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-21T23:07:05.537802+0000","dead_epoch":43},{"osd":7,"down_stamp":"2026-02-21T23:08:05.005884+0000","laggy_probability":0.30000001192092896,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-21T23:07:05.553574+0000","dead_epoch":43},{"osd":8,"down_stamp":"2026-02-21T23:08:05.005884+0000","laggy_probability":0.50999999046325684,"laggy_interval":1,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-21T23:07:05.535732+0000","dead_epoch":46},{"osd":9,"down_stamp":"2026-02-21T23:07:57.993298+0000","laggy_probability":0.30000001192092896,"laggy_interval":1,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"2026-02-21T23:07:05.536298+0000","dead_epoch":36}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"isa","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2026-02-21T23:08:26.714 INFO:tasks.ceph:Scrubbing osd.0 2026-02-21T23:08:26.714 DEBUG:teuthology.orchestra.run.trial150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2026-02-21T23:08:26.796 INFO:teuthology.orchestra.run.trial150.stdout:{ 2026-02-21T23:08:26.796 INFO:teuthology.orchestra.run.trial150.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2026-02-21T23:08:26.796 INFO:teuthology.orchestra.run.trial150.stdout:} 2026-02-21T23:08:26.802 DEBUG:teuthology.orchestra.run.trial150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2026-02-21T23:08:26.986 INFO:teuthology.orchestra.run.trial150.stderr:instructed osd(s) 0 to deep-scrub 2026-02-21T23:08:26.995 INFO:tasks.ceph:Scrubbing osd.1 2026-02-21T23:08:26.995 DEBUG:teuthology.orchestra.run.trial150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2026-02-21T23:08:27.078 INFO:teuthology.orchestra.run.trial150.stdout:{ 2026-02-21T23:08:27.078 INFO:teuthology.orchestra.run.trial150.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2026-02-21T23:08:27.078 INFO:teuthology.orchestra.run.trial150.stdout:} 2026-02-21T23:08:27.085 DEBUG:teuthology.orchestra.run.trial150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2026-02-21T23:08:27.263 INFO:teuthology.orchestra.run.trial150.stderr:instructed osd(s) 1 to deep-scrub 2026-02-21T23:08:27.272 INFO:tasks.ceph:Scrubbing osd.2 2026-02-21T23:08:27.272 DEBUG:teuthology.orchestra.run.trial150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2026-02-21T23:08:27.352 INFO:teuthology.orchestra.run.trial150.stdout:{ 2026-02-21T23:08:27.352 INFO:teuthology.orchestra.run.trial150.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2026-02-21T23:08:27.352 INFO:teuthology.orchestra.run.trial150.stdout:} 2026-02-21T23:08:27.358 DEBUG:teuthology.orchestra.run.trial150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2026-02-21T23:08:27.532 INFO:teuthology.orchestra.run.trial150.stderr:instructed osd(s) 2 to deep-scrub 2026-02-21T23:08:27.540 INFO:tasks.ceph:Scrubbing osd.3 2026-02-21T23:08:27.541 DEBUG:teuthology.orchestra.run.trial150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2026-02-21T23:08:27.621 INFO:teuthology.orchestra.run.trial150.stdout:{ 2026-02-21T23:08:27.622 INFO:teuthology.orchestra.run.trial150.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2026-02-21T23:08:27.622 INFO:teuthology.orchestra.run.trial150.stdout:} 2026-02-21T23:08:27.627 DEBUG:teuthology.orchestra.run.trial150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2026-02-21T23:08:27.804 INFO:teuthology.orchestra.run.trial150.stderr:instructed osd(s) 3 to deep-scrub 2026-02-21T23:08:27.813 INFO:tasks.ceph:Scrubbing osd.4 2026-02-21T23:08:27.813 DEBUG:teuthology.orchestra.run.trial150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2026-02-21T23:08:27.893 INFO:teuthology.orchestra.run.trial150.stdout:{ 2026-02-21T23:08:27.893 INFO:teuthology.orchestra.run.trial150.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2026-02-21T23:08:27.893 INFO:teuthology.orchestra.run.trial150.stdout:} 2026-02-21T23:08:27.898 DEBUG:teuthology.orchestra.run.trial150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2026-02-21T23:08:28.076 INFO:teuthology.orchestra.run.trial150.stderr:instructed osd(s) 4 to deep-scrub 2026-02-21T23:08:28.085 INFO:tasks.ceph:Scrubbing osd.5 2026-02-21T23:08:28.085 DEBUG:teuthology.orchestra.run.trial150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2026-02-21T23:08:28.166 INFO:teuthology.orchestra.run.trial150.stdout:{ 2026-02-21T23:08:28.166 INFO:teuthology.orchestra.run.trial150.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2026-02-21T23:08:28.166 INFO:teuthology.orchestra.run.trial150.stdout:} 2026-02-21T23:08:28.171 DEBUG:teuthology.orchestra.run.trial150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2026-02-21T23:08:28.349 INFO:teuthology.orchestra.run.trial150.stderr:instructed osd(s) 5 to deep-scrub 2026-02-21T23:08:28.358 INFO:tasks.ceph:Scrubbing osd.6 2026-02-21T23:08:28.358 DEBUG:teuthology.orchestra.run.trial150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2026-02-21T23:08:28.439 INFO:teuthology.orchestra.run.trial150.stdout:{ 2026-02-21T23:08:28.439 INFO:teuthology.orchestra.run.trial150.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2026-02-21T23:08:28.439 INFO:teuthology.orchestra.run.trial150.stdout:} 2026-02-21T23:08:28.444 DEBUG:teuthology.orchestra.run.trial150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2026-02-21T23:08:28.621 INFO:teuthology.orchestra.run.trial150.stderr:instructed osd(s) 6 to deep-scrub 2026-02-21T23:08:28.631 INFO:tasks.ceph:Scrubbing osd.7 2026-02-21T23:08:28.632 DEBUG:teuthology.orchestra.run.trial150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2026-02-21T23:08:28.714 INFO:teuthology.orchestra.run.trial150.stdout:{ 2026-02-21T23:08:28.714 INFO:teuthology.orchestra.run.trial150.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2026-02-21T23:08:28.714 INFO:teuthology.orchestra.run.trial150.stdout:} 2026-02-21T23:08:28.720 DEBUG:teuthology.orchestra.run.trial150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2026-02-21T23:08:28.898 INFO:teuthology.orchestra.run.trial150.stderr:instructed osd(s) 7 to deep-scrub 2026-02-21T23:08:28.908 INFO:tasks.ceph:Scrubbing osd.8 2026-02-21T23:08:28.909 DEBUG:teuthology.orchestra.run.trial150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.8 config set osd_debug_deep_scrub_sleep 0 2026-02-21T23:08:28.993 INFO:teuthology.orchestra.run.trial150.stdout:{ 2026-02-21T23:08:28.993 INFO:teuthology.orchestra.run.trial150.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2026-02-21T23:08:28.993 INFO:teuthology.orchestra.run.trial150.stdout:} 2026-02-21T23:08:28.999 DEBUG:teuthology.orchestra.run.trial150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 8 2026-02-21T23:08:29.177 INFO:teuthology.orchestra.run.trial150.stderr:instructed osd(s) 8 to deep-scrub 2026-02-21T23:08:29.187 INFO:tasks.ceph:Scrubbing osd.9 2026-02-21T23:08:29.187 DEBUG:teuthology.orchestra.run.trial150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.9 config set osd_debug_deep_scrub_sleep 0 2026-02-21T23:08:29.269 INFO:teuthology.orchestra.run.trial150.stdout:{ 2026-02-21T23:08:29.269 INFO:teuthology.orchestra.run.trial150.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2026-02-21T23:08:29.269 INFO:teuthology.orchestra.run.trial150.stdout:} 2026-02-21T23:08:29.278 DEBUG:teuthology.orchestra.run.trial150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 9 2026-02-21T23:08:29.453 INFO:teuthology.orchestra.run.trial150.stderr:instructed osd(s) 9 to deep-scrub 2026-02-21T23:08:29.462 DEBUG:teuthology.orchestra.run.trial150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T23:08:29.642 INFO:teuthology.orchestra.run.trial150.stdout: 2026-02-21T23:08:29.643 INFO:teuthology.orchestra.run.trial150.stderr:dumped all 2026-02-21T23:08:29.652 INFO:teuthology.orchestra.run.trial150.stdout:{"pg_ready":true,"pg_map":{"version":100,"stamp":"2026-02-21T23:08:28.822960+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":10,"num_per_pool_osds":10,"num_per_pool_omap_osds":8,"kb":1048576000,"kb_used":276208,"kb_used_data":6608,"kb_used_omap":258,"kb_used_meta":269181,"kb_avail":1048299792,"statfs":{"total":1073741824000,"available":1073458987008,"internally_reserved":0,"allocated":6766592,"data_stored":3275338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":264396,"internal_metadata":275642164},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.784300"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":73,"reported_epoch":59,"state":"active+clean","last_fresh":"2026-02-21T23:08:21.041254+0000","last_change":"2026-02-21T23:08:09.021397+0000","last_active":"2026-02-21T23:08:21.041254+0000","last_peered":"2026-02-21T23:08:21.041254+0000","last_clean":"2026-02-21T23:08:21.041254+0000","last_became_active":"2026-02-21T23:08:09.021232+0000","last_became_peered":"2026-02-21T23:08:09.021232+0000","last_unstale":"2026-02-21T23:08:21.041254+0000","last_undegraded":"2026-02-21T23:08:21.041254+0000","last_fullsized":"2026-02-21T23:08:21.041254+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T23:07:12.930820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T23:07:12.930820+0000","last_clean_scrub_stamp":"2026-02-21T23:07:12.930820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T01:01:37.847843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00033767299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":98,"reported_epoch":59,"state":"active+clean","last_fresh":"2026-02-21T23:08:21.040673+0000","last_change":"2026-02-21T23:08:09.021642+0000","last_active":"2026-02-21T23:08:21.040673+0000","last_peered":"2026-02-21T23:08:21.040673+0000","last_clean":"2026-02-21T23:08:21.040673+0000","last_became_active":"2026-02-21T23:08:09.021460+0000","last_became_peered":"2026-02-21T23:08:09.021460+0000","last_unstale":"2026-02-21T23:08:21.040673+0000","last_undegraded":"2026-02-21T23:08:21.040673+0000","last_fullsized":"2026-02-21T23:08:21.040673+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T23:07:12.930820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T23:07:12.930820+0000","last_clean_scrub_stamp":"2026-02-21T23:07:12.930820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T01:01:37.847843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00022801700000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":107,"reported_epoch":59,"state":"active+clean","last_fresh":"2026-02-21T23:08:21.633705+0000","last_change":"2026-02-21T23:08:20.150609+0000","last_active":"2026-02-21T23:08:21.633705+0000","last_peered":"2026-02-21T23:08:21.633705+0000","last_clean":"2026-02-21T23:08:21.633705+0000","last_became_active":"2026-02-21T23:08:20.150454+0000","last_became_peered":"2026-02-21T23:08:20.150454+0000","last_unstale":"2026-02-21T23:08:21.633705+0000","last_undegraded":"2026-02-21T23:08:21.633705+0000","last_fullsized":"2026-02-21T23:08:21.633705+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T23:07:12.930820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T23:07:12.930820+0000","last_clean_scrub_stamp":"2026-02-21T23:07:12.930820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T02:18:46.454644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00029313000000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0],"acting":[9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":78,"reported_epoch":59,"state":"active+clean","last_fresh":"2026-02-21T23:08:21.041583+0000","last_change":"2026-02-21T23:08:21.041523+0000","last_active":"2026-02-21T23:08:21.041583+0000","last_peered":"2026-02-21T23:08:21.041583+0000","last_clean":"2026-02-21T23:08:21.041583+0000","last_became_active":"2026-02-21T23:08:21.041355+0000","last_became_peered":"2026-02-21T23:08:21.041355+0000","last_unstale":"2026-02-21T23:08:21.041583+0000","last_undegraded":"2026-02-21T23:08:21.041583+0000","last_fullsized":"2026-02-21T23:08:21.041583+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T23:07:12.930820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T23:07:12.930820+0000","last_clean_scrub_stamp":"2026-02-21T23:07:12.930820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T07:10:36.306892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.3","version":"18'1","reported_seq":94,"reported_epoch":59,"state":"active+clean","last_fresh":"2026-02-21T23:08:21.041976+0000","last_change":"2026-02-21T23:08:21.041938+0000","last_active":"2026-02-21T23:08:21.041976+0000","last_peered":"2026-02-21T23:08:21.041976+0000","last_clean":"2026-02-21T23:08:21.041976+0000","last_became_active":"2026-02-21T23:08:21.041805+0000","last_became_peered":"2026-02-21T23:08:21.041805+0000","last_unstale":"2026-02-21T23:08:21.041976+0000","last_undegraded":"2026-02-21T23:08:21.041976+0000","last_fullsized":"2026-02-21T23:08:21.041976+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T23:07:12.930820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T23:07:12.930820+0000","last_clean_scrub_stamp":"2026-02-21T23:07:12.930820+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T06:30:21.881362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.2","version":"20'2","reported_seq":103,"reported_epoch":59,"state":"active+clean","last_fresh":"2026-02-21T23:08:21.041702+0000","last_change":"2026-02-21T23:08:21.041641+0000","last_active":"2026-02-21T23:08:21.041702+0000","last_peered":"2026-02-21T23:08:21.041702+0000","last_clean":"2026-02-21T23:08:21.041702+0000","last_became_active":"2026-02-21T23:08:21.041420+0000","last_became_peered":"2026-02-21T23:08:21.041420+0000","last_unstale":"2026-02-21T23:08:21.041702+0000","last_undegraded":"2026-02-21T23:08:21.041702+0000","last_fullsized":"2026-02-21T23:08:21.041702+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T23:07:12.930820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T23:07:12.930820+0000","last_clean_scrub_stamp":"2026-02-21T23:07:12.930820+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T09:14:00.344365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044592600000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":77,"reported_epoch":59,"state":"active+clean","last_fresh":"2026-02-21T23:08:21.040718+0000","last_change":"2026-02-21T23:08:09.021344+0000","last_active":"2026-02-21T23:08:21.040718+0000","last_peered":"2026-02-21T23:08:21.040718+0000","last_clean":"2026-02-21T23:08:21.040718+0000","last_became_active":"2026-02-21T23:08:09.021252+0000","last_became_peered":"2026-02-21T23:08:09.021252+0000","last_unstale":"2026-02-21T23:08:21.040718+0000","last_undegraded":"2026-02-21T23:08:21.040718+0000","last_fullsized":"2026-02-21T23:08:21.040718+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T23:07:12.930820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T23:07:12.930820+0000","last_clean_scrub_stamp":"2026-02-21T23:07:12.930820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T05:30:42.606649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00037064500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":85,"reported_epoch":59,"state":"active+clean","last_fresh":"2026-02-21T23:08:21.040761+0000","last_change":"2026-02-21T23:08:09.021203+0000","last_active":"2026-02-21T23:08:21.040761+0000","last_peered":"2026-02-21T23:08:21.040761+0000","last_clean":"2026-02-21T23:08:21.040761+0000","last_became_active":"2026-02-21T23:08:09.021076+0000","last_became_peered":"2026-02-21T23:08:09.021076+0000","last_unstale":"2026-02-21T23:08:21.040761+0000","last_undegraded":"2026-02-21T23:08:21.040761+0000","last_fullsized":"2026-02-21T23:08:21.040761+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T23:07:12.930820+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T23:07:12.930820+0000","last_clean_scrub_stamp":"2026-02-21T23:07:12.930820+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T00:21:40.020332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9],"acting":[5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":9}],"osd_stats":[{"osd":9,"up_from":40,"seq":171798691858,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":27620,"kb_used_data":660,"kb_used_omap":30,"kb_used_meta":26913,"kb_avail":104829980,"statfs":{"total":107374182400,"available":107345899520,"internally_reserved":0,"allocated":675840,"data_stored":327530,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":31001,"internal_metadata":27559655},"hb_peers":[0,1,2,3,4,5,6,7,8],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":47,"seq":201863462931,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":104857600,"kb_used":27620,"kb_used_data":660,"kb_used_omap":25,"kb_used_meta":26918,"kb_avail":104829980,"statfs":{"total":107374182400,"available":107345899520,"internally_reserved":0,"allocated":675840,"data_stored":327530,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":26440,"internal_metadata":27564216},"hb_peers":[0,1,2,3,4,5,6,7,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":46,"seq":197568495635,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":27620,"kb_used_data":660,"kb_used_omap":11,"kb_used_meta":26932,"kb_avail":104829980,"statfs":{"total":107374182400,"available":107345899520,"internally_reserved":0,"allocated":675840,"data_stored":327530,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12113,"internal_metadata":27578543},"hb_peers":[0,1,2,3,4,5,6,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495634,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":27620,"kb_used_data":660,"kb_used_omap":28,"kb_used_meta":26915,"kb_avail":104829980,"statfs":{"total":107374182400,"available":107345899520,"internally_reserved":0,"allocated":675840,"data_stored":327530,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":29043,"internal_metadata":27561613},"hb_peers":[0,1,2,3,4,5,7,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":46,"seq":197568495635,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":27620,"kb_used_data":660,"kb_used_omap":27,"kb_used_meta":26916,"kb_avail":104829980,"statfs":{"total":107374182400,"available":107345899520,"internally_reserved":0,"allocated":675840,"data_stored":327530,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":27742,"internal_metadata":27562914},"hb_peers":[0,1,2,3,4,6,7,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":46,"seq":197568495634,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":27624,"kb_used_data":664,"kb_used_omap":30,"kb_used_meta":26913,"kb_avail":104829976,"statfs":{"total":107374182400,"available":107345895424,"internally_reserved":0,"allocated":679936,"data_stored":327549,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":30997,"internal_metadata":27559659},"hb_peers":[0,1,2,3,5,6,7,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":46,"seq":197568495634,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":27620,"kb_used_data":660,"kb_used_omap":22,"kb_used_meta":26921,"kb_avail":104829980,"statfs":{"total":107374182400,"available":107345899520,"internally_reserved":0,"allocated":675840,"data_stored":327530,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":23183,"internal_metadata":27567473},"hb_peers":[0,1,2,4,5,6,7,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":46,"seq":197568495635,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":104857600,"kb_used":27620,"kb_used_data":660,"kb_used_omap":28,"kb_used_meta":26915,"kb_avail":104829980,"statfs":{"total":107374182400,"available":107345899520,"internally_reserved":0,"allocated":675840,"data_stored":327530,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":29044,"internal_metadata":27561612},"hb_peers":[0,1,3,4,5,6,7,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":58,"seq":249108103187,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":27624,"kb_used_data":664,"kb_used_omap":25,"kb_used_meta":26918,"kb_avail":104829976,"statfs":{"total":107374182400,"available":107345895424,"internally_reserved":0,"allocated":679936,"data_stored":327549,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":25789,"internal_metadata":27564867},"hb_peers":[0,2,3,4,5,6,7,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":57,"seq":244813135891,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":27620,"kb_used_data":660,"kb_used_omap":28,"kb_used_meta":26915,"kb_avail":104829980,"statfs":{"total":107374182400,"available":107345899520,"internally_reserved":0,"allocated":675840,"data_stored":327530,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":29044,"internal_metadata":27561612},"hb_peers":[1,2,3,4,5,6,7,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T23:08:29.657 INFO:tasks.ceph:pgid 1.7 last_scrub_stamp 2026-02-21T23:07:12.930820+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=21, tm_hour=23, tm_min=7, tm_sec=12, tm_wday=5, tm_yday=52, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=21, tm_hour=23, tm_min=8, tm_sec=25, tm_wday=5, tm_yday=52, tm_isdst=0) 2026-02-21T23:08:29.657 INFO:tasks.ceph:pgid 1.6 last_scrub_stamp 2026-02-21T23:07:12.930820+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=21, tm_hour=23, tm_min=7, tm_sec=12, tm_wday=5, tm_yday=52, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=21, tm_hour=23, tm_min=8, tm_sec=25, tm_wday=5, tm_yday=52, tm_isdst=0) 2026-02-21T23:08:29.657 INFO:tasks.ceph:pgid 1.5 last_scrub_stamp 2026-02-21T23:07:12.930820+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=21, tm_hour=23, tm_min=7, tm_sec=12, tm_wday=5, tm_yday=52, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=21, tm_hour=23, tm_min=8, tm_sec=25, tm_wday=5, tm_yday=52, tm_isdst=0) 2026-02-21T23:08:29.657 INFO:tasks.ceph:pgid 1.4 last_scrub_stamp 2026-02-21T23:07:12.930820+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=21, tm_hour=23, tm_min=7, tm_sec=12, tm_wday=5, tm_yday=52, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=21, tm_hour=23, tm_min=8, tm_sec=25, tm_wday=5, tm_yday=52, tm_isdst=0) 2026-02-21T23:08:29.657 INFO:tasks.ceph:pgid 1.3 last_scrub_stamp 2026-02-21T23:07:12.930820+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=21, tm_hour=23, tm_min=7, tm_sec=12, tm_wday=5, tm_yday=52, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=21, tm_hour=23, tm_min=8, tm_sec=25, tm_wday=5, tm_yday=52, tm_isdst=0) 2026-02-21T23:08:29.658 INFO:tasks.ceph:pgid 1.2 last_scrub_stamp 2026-02-21T23:07:12.930820+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=21, tm_hour=23, tm_min=7, tm_sec=12, tm_wday=5, tm_yday=52, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=21, tm_hour=23, tm_min=8, tm_sec=25, tm_wday=5, tm_yday=52, tm_isdst=0) 2026-02-21T23:08:29.658 INFO:tasks.ceph:pgid 1.1 last_scrub_stamp 2026-02-21T23:07:12.930820+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=21, tm_hour=23, tm_min=7, tm_sec=12, tm_wday=5, tm_yday=52, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=21, tm_hour=23, tm_min=8, tm_sec=25, tm_wday=5, tm_yday=52, tm_isdst=0) 2026-02-21T23:08:29.658 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2026-02-21T23:07:12.930820+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=21, tm_hour=23, tm_min=7, tm_sec=12, tm_wday=5, tm_yday=52, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=21, tm_hour=23, tm_min=8, tm_sec=25, tm_wday=5, tm_yday=52, tm_isdst=0) 2026-02-21T23:08:29.658 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2026-02-21T23:08:49.659 DEBUG:teuthology.orchestra.run.trial150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T23:08:49.837 INFO:teuthology.orchestra.run.trial150.stdout: 2026-02-21T23:08:49.837 INFO:teuthology.orchestra.run.trial150.stderr:dumped all 2026-02-21T23:08:49.846 INFO:teuthology.orchestra.run.trial150.stdout:{"pg_ready":true,"pg_map":{"version":110,"stamp":"2026-02-21T23:08:48.824636+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5,"ondisk_log_size":5,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":10,"num_per_pool_osds":10,"num_per_pool_omap_osds":8,"kb":1048576000,"kb_used":276208,"kb_used_data":6608,"kb_used_omap":258,"kb_used_meta":269181,"kb_avail":1048299792,"statfs":{"total":1073741824000,"available":1073458987008,"internally_reserved":0,"allocated":6766592,"data_stored":3275338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":264396,"internal_metadata":275642164},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.000921"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":81,"reported_epoch":59,"state":"active+clean","last_fresh":"2026-02-21T23:08:29.170919+0000","last_change":"2026-02-21T23:08:29.170919+0000","last_active":"2026-02-21T23:08:29.170919+0000","last_peered":"2026-02-21T23:08:29.170919+0000","last_clean":"2026-02-21T23:08:29.170919+0000","last_became_active":"2026-02-21T23:08:09.021232+0000","last_became_peered":"2026-02-21T23:08:09.021232+0000","last_unstale":"2026-02-21T23:08:29.170919+0000","last_undegraded":"2026-02-21T23:08:29.170919+0000","last_fullsized":"2026-02-21T23:08:29.170919+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T23:08:29.170887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T23:08:29.170887+0000","last_clean_scrub_stamp":"2026-02-21T23:08:29.170887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T07:58:46.914676+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0.00033767299999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":106,"reported_epoch":59,"state":"active+clean","last_fresh":"2026-02-21T23:08:29.279701+0000","last_change":"2026-02-21T23:08:29.279701+0000","last_active":"2026-02-21T23:08:29.279701+0000","last_peered":"2026-02-21T23:08:29.279701+0000","last_clean":"2026-02-21T23:08:29.279701+0000","last_became_active":"2026-02-21T23:08:09.021460+0000","last_became_peered":"2026-02-21T23:08:09.021460+0000","last_unstale":"2026-02-21T23:08:29.279701+0000","last_undegraded":"2026-02-21T23:08:29.279701+0000","last_fullsized":"2026-02-21T23:08:29.279701+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T23:08:29.279673+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T23:08:29.279673+0000","last_clean_scrub_stamp":"2026-02-21T23:08:29.279673+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T11:05:48.030155+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0.00022801700000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":115,"reported_epoch":59,"state":"active+clean","last_fresh":"2026-02-21T23:08:29.712875+0000","last_change":"2026-02-21T23:08:29.712875+0000","last_active":"2026-02-21T23:08:29.712875+0000","last_peered":"2026-02-21T23:08:29.712875+0000","last_clean":"2026-02-21T23:08:29.712875+0000","last_became_active":"2026-02-21T23:08:20.150454+0000","last_became_peered":"2026-02-21T23:08:20.150454+0000","last_unstale":"2026-02-21T23:08:29.712875+0000","last_undegraded":"2026-02-21T23:08:29.712875+0000","last_fullsized":"2026-02-21T23:08:29.712875+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T23:08:29.712843+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T23:08:29.712843+0000","last_clean_scrub_stamp":"2026-02-21T23:08:29.712843+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T03:00:33.331473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00029313000000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0],"acting":[9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":86,"reported_epoch":59,"state":"active+clean","last_fresh":"2026-02-21T23:08:27.309011+0000","last_change":"2026-02-21T23:08:27.309011+0000","last_active":"2026-02-21T23:08:27.309011+0000","last_peered":"2026-02-21T23:08:27.309011+0000","last_clean":"2026-02-21T23:08:27.309011+0000","last_became_active":"2026-02-21T23:08:21.041355+0000","last_became_peered":"2026-02-21T23:08:21.041355+0000","last_unstale":"2026-02-21T23:08:27.309011+0000","last_undegraded":"2026-02-21T23:08:27.309011+0000","last_fullsized":"2026-02-21T23:08:27.309011+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T23:08:27.308978+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T23:08:27.308978+0000","last_clean_scrub_stamp":"2026-02-21T23:08:27.308978+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T07:07:56.401890+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.3","version":"59'2","reported_seq":103,"reported_epoch":59,"state":"active+clean","last_fresh":"2026-02-21T23:08:28.180542+0000","last_change":"2026-02-21T23:08:28.180542+0000","last_active":"2026-02-21T23:08:28.180542+0000","last_peered":"2026-02-21T23:08:28.180542+0000","last_clean":"2026-02-21T23:08:28.180542+0000","last_became_active":"2026-02-21T23:08:21.041805+0000","last_became_peered":"2026-02-21T23:08:21.041805+0000","last_unstale":"2026-02-21T23:08:28.180542+0000","last_undegraded":"2026-02-21T23:08:28.180542+0000","last_fullsized":"2026-02-21T23:08:28.180542+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'2","last_scrub_stamp":"2026-02-21T23:08:28.180512+0000","last_deep_scrub":"59'2","last_deep_scrub_stamp":"2026-02-21T23:08:28.180512+0000","last_clean_scrub_stamp":"2026-02-21T23:08:28.180512+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T11:05:46.930995+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.2","version":"59'3","reported_seq":112,"reported_epoch":59,"state":"active+clean","last_fresh":"2026-02-21T23:08:28.986029+0000","last_change":"2026-02-21T23:08:28.986029+0000","last_active":"2026-02-21T23:08:28.986029+0000","last_peered":"2026-02-21T23:08:28.986029+0000","last_clean":"2026-02-21T23:08:28.986029+0000","last_became_active":"2026-02-21T23:08:21.041420+0000","last_became_peered":"2026-02-21T23:08:21.041420+0000","last_unstale":"2026-02-21T23:08:28.986029+0000","last_undegraded":"2026-02-21T23:08:28.986029+0000","last_fullsized":"2026-02-21T23:08:28.986029+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"59'3","last_scrub_stamp":"2026-02-21T23:08:28.985997+0000","last_deep_scrub":"59'3","last_deep_scrub_stamp":"2026-02-21T23:08:28.985997+0000","last_clean_scrub_stamp":"2026-02-21T23:08:28.985997+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T01:22:43.449193+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0.00044592600000000003,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":85,"reported_epoch":59,"state":"active+clean","last_fresh":"2026-02-21T23:08:29.031908+0000","last_change":"2026-02-21T23:08:29.031908+0000","last_active":"2026-02-21T23:08:29.031908+0000","last_peered":"2026-02-21T23:08:29.031908+0000","last_clean":"2026-02-21T23:08:29.031908+0000","last_became_active":"2026-02-21T23:08:09.021252+0000","last_became_peered":"2026-02-21T23:08:09.021252+0000","last_unstale":"2026-02-21T23:08:29.031908+0000","last_undegraded":"2026-02-21T23:08:29.031908+0000","last_fullsized":"2026-02-21T23:08:29.031908+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T23:08:29.031869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T23:08:29.031869+0000","last_clean_scrub_stamp":"2026-02-21T23:08:29.031869+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T07:14:03.456174+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0.00037064500000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":93,"reported_epoch":59,"state":"active+clean","last_fresh":"2026-02-21T23:08:28.476848+0000","last_change":"2026-02-21T23:08:28.476848+0000","last_active":"2026-02-21T23:08:28.476848+0000","last_peered":"2026-02-21T23:08:28.476848+0000","last_clean":"2026-02-21T23:08:28.476848+0000","last_became_active":"2026-02-21T23:08:09.021076+0000","last_became_peered":"2026-02-21T23:08:09.021076+0000","last_unstale":"2026-02-21T23:08:28.476848+0000","last_undegraded":"2026-02-21T23:08:28.476848+0000","last_fullsized":"2026-02-21T23:08:28.476848+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T23:08:28.476811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T23:08:28.476811+0000","last_clean_scrub_stamp":"2026-02-21T23:08:28.476811+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2026-02-23T08:07:30.789813+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9],"acting":[5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5,"ondisk_log_size":5,"up":16,"acting":16,"num_store_stats":9}],"osd_stats":[{"osd":9,"up_from":40,"seq":171798691862,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":27620,"kb_used_data":660,"kb_used_omap":30,"kb_used_meta":26913,"kb_avail":104829980,"statfs":{"total":107374182400,"available":107345899520,"internally_reserved":0,"allocated":675840,"data_stored":327530,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":31001,"internal_metadata":27559655},"hb_peers":[0,1,2,3,4,5,6,7,8],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":47,"seq":201863462935,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":104857600,"kb_used":27620,"kb_used_data":660,"kb_used_omap":25,"kb_used_meta":26918,"kb_avail":104829980,"statfs":{"total":107374182400,"available":107345899520,"internally_reserved":0,"allocated":675840,"data_stored":327530,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":26440,"internal_metadata":27564216},"hb_peers":[0,1,2,3,4,5,6,7,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":46,"seq":197568495639,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":27620,"kb_used_data":660,"kb_used_omap":11,"kb_used_meta":26932,"kb_avail":104829980,"statfs":{"total":107374182400,"available":107345899520,"internally_reserved":0,"allocated":675840,"data_stored":327530,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12113,"internal_metadata":27578543},"hb_peers":[0,1,2,3,4,5,6,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495638,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":27620,"kb_used_data":660,"kb_used_omap":28,"kb_used_meta":26915,"kb_avail":104829980,"statfs":{"total":107374182400,"available":107345899520,"internally_reserved":0,"allocated":675840,"data_stored":327530,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":29043,"internal_metadata":27561613},"hb_peers":[0,1,2,3,4,5,7,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":46,"seq":197568495639,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":27620,"kb_used_data":660,"kb_used_omap":27,"kb_used_meta":26916,"kb_avail":104829980,"statfs":{"total":107374182400,"available":107345899520,"internally_reserved":0,"allocated":675840,"data_stored":327530,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":27742,"internal_metadata":27562914},"hb_peers":[0,1,2,3,4,6,7,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":46,"seq":197568495638,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":27624,"kb_used_data":664,"kb_used_omap":30,"kb_used_meta":26913,"kb_avail":104829976,"statfs":{"total":107374182400,"available":107345895424,"internally_reserved":0,"allocated":679936,"data_stored":327549,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":30997,"internal_metadata":27559659},"hb_peers":[0,1,2,3,5,6,7,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":46,"seq":197568495638,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":27620,"kb_used_data":660,"kb_used_omap":22,"kb_used_meta":26921,"kb_avail":104829980,"statfs":{"total":107374182400,"available":107345899520,"internally_reserved":0,"allocated":675840,"data_stored":327530,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":23183,"internal_metadata":27567473},"hb_peers":[0,1,2,4,5,6,7,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":46,"seq":197568495639,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":104857600,"kb_used":27620,"kb_used_data":660,"kb_used_omap":28,"kb_used_meta":26915,"kb_avail":104829980,"statfs":{"total":107374182400,"available":107345899520,"internally_reserved":0,"allocated":675840,"data_stored":327530,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":29044,"internal_metadata":27561612},"hb_peers":[0,1,3,4,5,6,7,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":58,"seq":249108103191,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":27624,"kb_used_data":664,"kb_used_omap":25,"kb_used_meta":26918,"kb_avail":104829976,"statfs":{"total":107374182400,"available":107345895424,"internally_reserved":0,"allocated":679936,"data_stored":327549,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":25789,"internal_metadata":27564867},"hb_peers":[0,2,3,4,5,6,7,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":57,"seq":244813135895,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":27620,"kb_used_data":660,"kb_used_omap":28,"kb_used_meta":26915,"kb_avail":104829980,"statfs":{"total":107374182400,"available":107345899520,"internally_reserved":0,"allocated":675840,"data_stored":327530,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":29044,"internal_metadata":27561612},"hb_peers":[1,2,3,4,5,6,7,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T23:08:49.846 DEBUG:teuthology.orchestra.run.trial150:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2026-02-21T23:08:50.034 INFO:teuthology.misc:Shutting down mds daemons... 2026-02-21T23:08:50.034 INFO:teuthology.misc:Shutting down osd daemons... 2026-02-21T23:08:50.034 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2026-02-21T23:08:50.034 INFO:teuthology.orchestra.run:waiting for 300 2026-02-21T23:08:50.073 INFO:tasks.ceph.osd.0:Stopped 2026-02-21T23:08:50.073 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2026-02-21T23:08:50.073 INFO:teuthology.orchestra.run:waiting for 300 2026-02-21T23:08:50.148 INFO:tasks.ceph.osd.1:Stopped 2026-02-21T23:08:50.148 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2026-02-21T23:08:50.148 INFO:teuthology.orchestra.run:waiting for 300 2026-02-21T23:08:50.219 INFO:tasks.ceph.osd.2:Stopped 2026-02-21T23:08:50.220 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2026-02-21T23:08:50.220 INFO:teuthology.orchestra.run:waiting for 300 2026-02-21T23:08:50.297 INFO:tasks.ceph.osd.3:Stopped 2026-02-21T23:08:50.297 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2026-02-21T23:08:50.297 INFO:teuthology.orchestra.run:waiting for 300 2026-02-21T23:08:50.375 INFO:tasks.ceph.osd.4:Stopped 2026-02-21T23:08:50.375 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2026-02-21T23:08:50.375 INFO:teuthology.orchestra.run:waiting for 300 2026-02-21T23:08:50.448 INFO:tasks.ceph.osd.5:Stopped 2026-02-21T23:08:50.448 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2026-02-21T23:08:50.449 INFO:teuthology.orchestra.run:waiting for 300 2026-02-21T23:08:50.517 INFO:tasks.ceph.osd.6:Stopped 2026-02-21T23:08:50.518 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2026-02-21T23:08:50.518 INFO:teuthology.orchestra.run:waiting for 300 2026-02-21T23:08:50.588 INFO:tasks.ceph.osd.7:Stopped 2026-02-21T23:08:50.589 DEBUG:tasks.ceph.osd.8:waiting for process to exit 2026-02-21T23:08:50.589 INFO:teuthology.orchestra.run:waiting for 300 2026-02-21T23:08:50.661 INFO:tasks.ceph.osd.8:Stopped 2026-02-21T23:08:50.661 DEBUG:tasks.ceph.osd.9:waiting for process to exit 2026-02-21T23:08:50.661 INFO:teuthology.orchestra.run:waiting for 300 2026-02-21T23:08:50.734 INFO:tasks.ceph.osd.9:Stopped 2026-02-21T23:08:50.734 INFO:teuthology.misc:Shutting down mgr daemons... 2026-02-21T23:08:50.735 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2026-02-21T23:08:50.735 INFO:teuthology.orchestra.run:waiting for 300 2026-02-21T23:08:50.792 INFO:tasks.ceph.mgr.x:Stopped 2026-02-21T23:08:50.792 INFO:teuthology.misc:Shutting down mon daemons... 2026-02-21T23:08:50.792 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2026-02-21T23:08:50.792 INFO:teuthology.orchestra.run:waiting for 300 2026-02-21T23:08:50.841 INFO:tasks.ceph.mon.a:Stopped 2026-02-21T23:08:50.841 INFO:tasks.ceph:Checking cluster log for badness... 2026-02-21T23:08:50.842 DEBUG:teuthology.orchestra.run.trial150:> sudo grep -E '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | grep -E -v 'but it is still running' | grep -E -v 'overall HEALTH_' | grep -E -v '\(OSDMAP_FLAGS\)' | grep -E -v '\(OSD_' | grep -E -v '\(PG_' | grep -E -v '\(POOL_APP_NOT_ENABLED\)' | grep -E -v '\(MDS_ALL_DOWN\)' | grep -E -v '\(MDS_UP_LESS_THAN_MAX\)' | head -n 1 2026-02-21T23:08:50.907 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@trial150.front.sepia.ceph.com 2026-02-21T23:08:50.907 DEBUG:teuthology.orchestra.run.trial150:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2026-02-21T23:08:50.984 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@trial150.front.sepia.ceph.com 2026-02-21T23:08:50.984 DEBUG:teuthology.orchestra.run.trial150:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2026-02-21T23:08:51.053 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@trial150.front.sepia.ceph.com 2026-02-21T23:08:51.053 DEBUG:teuthology.orchestra.run.trial150:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2026-02-21T23:08:51.120 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@trial150.front.sepia.ceph.com 2026-02-21T23:08:51.121 DEBUG:teuthology.orchestra.run.trial150:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2026-02-21T23:08:51.189 INFO:tasks.ceph:Archiving mon data... 2026-02-21T23:08:51.193 DEBUG:teuthology.misc:Transferring archived files from trial150:/var/lib/ceph/mon/ceph-a to /home/teuthworker/mnt/teuthology/nmordech-2026-02-21_21:55:34-rados-wip-rocky10-branch-of-the-day-2026-02-21-1771692874-tentacle-distro-default-trial/63256/data/mon.a.tgz 2026-02-21T23:08:51.194 DEBUG:teuthology.orchestra.run.trial150:> mktemp 2026-02-21T23:08:51.244 INFO:teuthology.orchestra.run.trial150.stdout:/tmp/tmp.btzDDHrjXw 2026-02-21T23:08:51.244 DEBUG:teuthology.orchestra.run.trial150:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.btzDDHrjXw 2026-02-21T23:08:51.335 DEBUG:teuthology.orchestra.run.trial150:> sudo chmod 0666 /tmp/tmp.btzDDHrjXw 2026-02-21T23:08:51.417 DEBUG:teuthology.orchestra.remote:trial150:/tmp/tmp.btzDDHrjXw is 215KB 2026-02-21T23:08:51.512 DEBUG:teuthology.orchestra.run.trial150:> rm -fr /tmp/tmp.btzDDHrjXw 2026-02-21T23:08:51.528 INFO:tasks.ceph:Cleaning ceph cluster... 2026-02-21T23:08:51.528 DEBUG:teuthology.orchestra.run.trial150:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2026-02-21T23:08:51.651 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2026-02-21T23:08:51.651 INFO:tasks.ceph:Archiving crash dumps... 2026-02-21T23:08:51.653 DEBUG:teuthology.misc:Transferring archived files from trial150:/var/lib/ceph/crash to /home/teuthworker/mnt/teuthology/nmordech-2026-02-21_21:55:34-rados-wip-rocky10-branch-of-the-day-2026-02-21-1771692874-tentacle-distro-default-trial/63256/remote/trial150/crash 2026-02-21T23:08:51.654 DEBUG:teuthology.orchestra.run.trial150:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2026-02-21T23:08:51.714 INFO:tasks.ceph:Compressing logs... 2026-02-21T23:08:51.714 DEBUG:teuthology.orchestra.run.trial150:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2026-02-21T23:08:51.778 INFO:teuthology.orchestra.run.trial150.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2026-02-21T23:08:51.778 INFO:teuthology.orchestra.run.trial150.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56002.log 2026-02-21T23:08:51.820 INFO:teuthology.orchestra.run.trial150.stderr:/var/log/ceph/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54182.log 2026-02-21T23:08:51.820 INFO:teuthology.orchestra.run.trial150.stderr:gzip -5/var/log/ceph/ceph-client.admin.56002.log: --verbose -- /var/log/ceph/ceph-mon.a.log 2026-02-21T23:08:51.820 INFO:teuthology.orchestra.run.trial150.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56002.log.gz 2026-02-21T23:08:51.820 INFO:teuthology.orchestra.run.trial150.stderr:/var/log/ceph/ceph-client.admin.54182.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54182.log.gz 2026-02-21T23:08:51.820 INFO:teuthology.orchestra.run.trial150.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57747.log 2026-02-21T23:08:51.820 INFO:teuthology.orchestra.run.trial150.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56931.log 2026-02-21T23:08:51.820 INFO:teuthology.orchestra.run.trial150.stderr:/var/log/ceph/ceph-mon.a.log: /var/log/ceph/ceph-client.admin.57747.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58140.log 2026-02-21T23:08:51.821 INFO:teuthology.orchestra.run.trial150.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57747.log.gz 2026-02-21T23:08:51.821 INFO:teuthology.orchestra.run.trial150.stderr:/var/log/ceph/ceph-client.admin.56931.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56931.log.gz 2026-02-21T23:08:51.821 INFO:teuthology.orchestra.run.trial150.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57500.log 2026-02-21T23:08:51.821 INFO:teuthology.orchestra.run.trial150.stderr:/var/log/ceph/ceph-client.admin.58140.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2026-02-21T23:08:51.821 INFO:teuthology.orchestra.run.trial150.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58140.log.gz 2026-02-21T23:08:51.821 INFO:teuthology.orchestra.run.trial150.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48523.log 2026-02-21T23:08:51.821 INFO:teuthology.orchestra.run.trial150.stderr:/var/log/ceph/ceph-client.admin.57500.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57500.log.gz 2026-02-21T23:08:51.821 INFO:teuthology.orchestra.run.trial150.stderr:/var/log/ceph/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58483.log 2026-02-21T23:08:51.821 INFO:teuthology.orchestra.run.trial150.stderr:/var/log/ceph/ceph-client.admin.48523.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54708.log 2026-02-21T23:08:51.822 INFO:teuthology.orchestra.run.trial150.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48523.log.gz 2026-02-21T23:08:51.822 INFO:teuthology.orchestra.run.trial150.stderr:/var/log/ceph/ceph-client.admin.58483.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56708.log 2026-02-21T23:08:51.822 INFO:teuthology.orchestra.run.trial150.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58483.log.gz 2026-02-21T23:08:51.822 INFO:teuthology.orchestra.run.trial150.stderr:/var/log/ceph/ceph-client.admin.54708.log: gzip -5 0.0% --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.54708.log.gz /var/log/ceph/ceph-client.admin.56456.log 2026-02-21T23:08:51.822 INFO:teuthology.orchestra.run.trial150.stderr: 2026-02-21T23:08:51.822 INFO:teuthology.orchestra.run.trial150.stderr:/var/log/ceph/ceph-client.admin.56708.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55718.log 2026-02-21T23:08:51.822 INFO:teuthology.orchestra.run.trial150.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56708.log.gz 2026-02-21T23:08:51.822 INFO:teuthology.orchestra.run.trial150.stderr:/var/log/ceph/ceph-client.admin.56456.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55069.log 2026-02-21T23:08:51.822 INFO:teuthology.orchestra.run.trial150.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56456.log.gz 2026-02-21T23:08:51.823 INFO:teuthology.orchestra.run.trial150.stderr:/var/log/ceph/ceph-client.admin.55718.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48476.log 2026-02-21T23:08:51.823 INFO:teuthology.orchestra.run.trial150.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55718.log.gz 2026-02-21T23:08:51.823 INFO:teuthology.orchestra.run.trial150.stderr:/var/log/ceph/ceph-client.admin.55069.log: gzip -5 --verbose -- 0.0% /var/log/ceph/ceph-client.admin.56530.log -- replaced with /var/log/ceph/ceph-client.admin.55069.log.gz 2026-02-21T23:08:51.823 INFO:teuthology.orchestra.run.trial150.stderr: 2026-02-21T23:08:51.823 INFO:teuthology.orchestra.run.trial150.stderr:/var/log/ceph/ceph-client.admin.48476.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57845.log 2026-02-21T23:08:51.823 INFO:teuthology.orchestra.run.trial150.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48476.log.gz 2026-02-21T23:08:51.823 INFO:teuthology.orchestra.run.trial150.stderr:/var/log/ceph/ceph-client.admin.56530.log: gzip -5 --verbose 0.0% -- -- replaced with /var/log/ceph/ceph-client.admin.56530.log.gz /var/log/ceph/ceph-osd.3.log 2026-02-21T23:08:51.823 INFO:teuthology.orchestra.run.trial150.stderr: 2026-02-21T23:08:51.823 INFO:teuthology.orchestra.run.trial150.stderr:/var/log/ceph/ceph-client.admin.57845.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58728.log 2026-02-21T23:08:51.824 INFO:teuthology.orchestra.run.trial150.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57845.log.gz 2026-02-21T23:08:51.824 INFO:teuthology.orchestra.run.trial150.stderr:/var/log/ceph/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56505.log 2026-02-21T23:08:51.824 INFO:teuthology.orchestra.run.trial150.stderr:/var/log/ceph/ceph-client.admin.58728.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54690.log 2026-02-21T23:08:51.824 INFO:teuthology.orchestra.run.trial150.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58728.log.gz 2026-02-21T23:08:51.824 INFO:teuthology.orchestra.run.trial150.stderr:/var/log/ceph/ceph-client.admin.56505.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54702.log 2026-02-21T23:08:51.824 INFO:teuthology.orchestra.run.trial150.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56505.log.gz 2026-02-21T23:08:51.824 INFO:teuthology.orchestra.run.trial150.stderr:/var/log/ceph/ceph-client.admin.54690.log: 0.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.54690.log.gz -5 2026-02-21T23:08:51.824 INFO:teuthology.orchestra.run.trial150.stderr: --verbose -- /var/log/ceph/ceph-client.admin.47737.log 2026-02-21T23:08:51.824 INFO:teuthology.orchestra.run.trial150.stderr:/var/log/ceph/ceph-client.admin.54702.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56556.log 2026-02-21T23:08:51.825 INFO:teuthology.orchestra.run.trial150.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54702.log.gz 2026-02-21T23:08:51.825 INFO:teuthology.orchestra.run.trial150.stderr:/var/log/ceph/ceph-client.admin.47737.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55445.log 2026-02-21T23:08:51.825 INFO:teuthology.orchestra.run.trial150.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47737.log.gz 2026-02-21T23:08:51.825 INFO:teuthology.orchestra.run.trial150.stderr:/var/log/ceph/ceph-client.admin.56556.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57350.log 2026-02-21T23:08:51.825 INFO:teuthology.orchestra.run.trial150.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56556.log.gz 2026-02-21T23:08:51.825 INFO:teuthology.orchestra.run.trial150.stderr:/var/log/ceph/ceph-client.admin.55445.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55445.log.gz 2026-02-21T23:08:51.825 INFO:teuthology.orchestra.run.trial150.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53683.log 2026-02-21T23:08:51.825 INFO:teuthology.orchestra.run.trial150.stderr:/var/log/ceph/ceph-client.admin.57350.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56658.log 2026-02-21T23:08:51.825 INFO:teuthology.orchestra.run.trial150.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57350.log.gz 2026-02-21T23:08:51.825 INFO:teuthology.orchestra.run.trial150.stderr:/var/log/ceph/ceph-client.admin.53683.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53683.log.gz 2026-02-21T23:08:51.826 INFO:teuthology.orchestra.run.trial150.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47808.log 2026-02-21T23:08:51.826 INFO:teuthology.orchestra.run.trial150.stderr:/var/log/ceph/ceph-client.admin.56658.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56658.log.gz 2026-02-21T23:08:51.826 INFO:teuthology.orchestra.run.trial150.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57254.log 2026-02-21T23:08:51.826 INFO:teuthology.orchestra.run.trial150.stderr:/var/log/ceph/ceph-client.admin.47808.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58826.log 2026-02-21T23:08:51.826 INFO:teuthology.orchestra.run.trial150.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47808.log.gz 2026-02-21T23:08:51.826 INFO:teuthology.orchestra.run.trial150.stderr:/var/log/ceph/ceph-client.admin.57254.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57254.log.gz 2026-02-21T23:08:51.826 INFO:teuthology.orchestra.run.trial150.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58385.log 2026-02-21T23:08:51.826 INFO:teuthology.orchestra.run.trial150.stderr:/var/log/ceph/ceph-client.admin.58826.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54179.log 2026-02-21T23:08:51.826 INFO:teuthology.orchestra.run.trial150.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58826.log.gz 2026-02-21T23:08:51.826 INFO:teuthology.orchestra.run.trial150.stderr:/var/log/ceph/ceph-client.admin.58385.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48570.log 2026-02-21T23:08:51.827 INFO:teuthology.orchestra.run.trial150.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58385.log.gz 2026-02-21T23:08:51.827 INFO:teuthology.orchestra.run.trial150.stderr:/var/log/ceph/ceph-client.admin.54179.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54179.log.gz 2026-02-21T23:08:51.827 INFO:teuthology.orchestra.run.trial150.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56907.log 2026-02-21T23:08:51.827 INFO:teuthology.orchestra.run.trial150.stderr:/var/log/ceph/ceph-client.admin.48570.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2026-02-21T23:08:51.827 INFO:teuthology.orchestra.run.trial150.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48570.log.gz 2026-02-21T23:08:51.827 INFO:teuthology.orchestra.run.trial150.stderr:/var/log/ceph/ceph-client.admin.56907.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.7.log 2026-02-21T23:08:51.827 INFO:teuthology.orchestra.run.trial150.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56907.log.gz 2026-02-21T23:08:51.827 INFO:teuthology.orchestra.run.trial150.stderr:/var/log/ceph/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48429.log 2026-02-21T23:08:51.827 INFO:teuthology.orchestra.run.trial150.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54173.log 2026-02-21T23:08:51.827 INFO:teuthology.orchestra.run.trial150.stderr:/var/log/ceph/ceph-osd.7.log: /var/log/ceph/ceph-client.admin.48429.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54187.log 2026-02-21T23:08:51.828 INFO:teuthology.orchestra.run.trial150.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48429.log.gz 2026-02-21T23:08:51.828 INFO:teuthology.orchestra.run.trial150.stderr:/var/log/ceph/ceph-client.admin.54173.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54173.log.gz 2026-02-21T23:08:51.828 INFO:teuthology.orchestra.run.trial150.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56383.log 2026-02-21T23:08:51.828 INFO:teuthology.orchestra.run.trial150.stderr:/var/log/ceph/ceph-client.admin.54187.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53658.log 2026-02-21T23:08:51.828 INFO:teuthology.orchestra.run.trial150.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54187.log.gz 2026-02-21T23:08:51.828 INFO:teuthology.orchestra.run.trial150.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56732.log 2026-02-21T23:08:51.828 INFO:teuthology.orchestra.run.trial150.stderr:/var/log/ceph/ceph-client.admin.56383.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56383.log.gz 2026-02-21T23:08:51.828 INFO:teuthology.orchestra.run.trial150.stderr:/var/log/ceph/ceph-client.admin.53658.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55904.log 2026-02-21T23:08:51.828 INFO:teuthology.orchestra.run.trial150.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53658.log.gz 2026-02-21T23:08:51.828 INFO:teuthology.orchestra.run.trial150.stderr:/var/log/ceph/ceph-client.admin.56732.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56732.log.gz 2026-02-21T23:08:51.829 INFO:teuthology.orchestra.run.trial150.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58924.log 2026-02-21T23:08:51.829 INFO:teuthology.orchestra.run.trial150.stderr:/var/log/ceph/ceph-client.admin.55904.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55904.log.gz 2026-02-21T23:08:51.829 INFO:teuthology.orchestra.run.trial150.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58777.log 2026-02-21T23:08:51.829 INFO:teuthology.orchestra.run.trial150.stderr:/var/log/ceph/ceph-client.admin.58924.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57006.log 2026-02-21T23:08:51.829 INFO:teuthology.orchestra.run.trial150.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58924.log.gz 2026-02-21T23:08:51.829 INFO:teuthology.orchestra.run.trial150.stderr:/var/log/ceph/ceph-client.admin.58777.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56831.log 2026-02-21T23:08:51.829 INFO:teuthology.orchestra.run.trial150.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58777.log.gz 2026-02-21T23:08:51.829 INFO:teuthology.orchestra.run.trial150.stderr:/var/log/ceph/ceph-client.admin.57006.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55707.log 2026-02-21T23:08:51.829 INFO:teuthology.orchestra.run.trial150.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57006.log.gz 2026-02-21T23:08:51.829 INFO:teuthology.orchestra.run.trial150.stderr:/var/log/ceph/ceph-client.admin.56831.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54175.log 2026-02-21T23:08:51.829 INFO:teuthology.orchestra.run.trial150.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56831.log.gz 2026-02-21T23:08:51.830 INFO:teuthology.orchestra.run.trial150.stderr:/var/log/ceph/ceph-client.admin.55707.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54186.log 2026-02-21T23:08:51.830 INFO:teuthology.orchestra.run.trial150.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55707.log.gz 2026-02-21T23:08:51.830 INFO:teuthology.orchestra.run.trial150.stderr:/var/log/ceph/ceph-client.admin.54175.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48617.log 2026-02-21T23:08:51.830 INFO:teuthology.orchestra.run.trial150.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54175.log.gz 2026-02-21T23:08:51.830 INFO:teuthology.orchestra.run.trial150.stderr:/var/log/ceph/ceph-client.admin.54186.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59022.log 2026-02-21T23:08:51.830 INFO:teuthology.orchestra.run.trial150.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54186.log.gz 2026-02-21T23:08:51.830 INFO:teuthology.orchestra.run.trial150.stderr:/var/log/ceph/ceph-client.admin.48617.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57079.log 2026-02-21T23:08:51.830 INFO:teuthology.orchestra.run.trial150.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48617.log.gz 2026-02-21T23:08:51.830 INFO:teuthology.orchestra.run.trial150.stderr:/var/log/ceph/ceph-client.admin.59022.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54714.log 2026-02-21T23:08:51.830 INFO:teuthology.orchestra.run.trial150.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59022.log.gz 2026-02-21T23:08:51.830 INFO:teuthology.orchestra.run.trial150.stderr:/var/log/ceph/ceph-client.admin.57079.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53588.log 2026-02-21T23:08:51.831 INFO:teuthology.orchestra.run.trial150.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57079.log.gz 2026-02-21T23:08:51.831 INFO:teuthology.orchestra.run.trial150.stderr:/var/log/ceph/ceph-client.admin.54714.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57375.log 2026-02-21T23:08:51.831 INFO:teuthology.orchestra.run.trial150.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54714.log.gz 2026-02-21T23:08:51.831 INFO:teuthology.orchestra.run.trial150.stderr:/var/log/ceph/ceph-client.admin.53588.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58336.log 2026-02-21T23:08:51.831 INFO:teuthology.orchestra.run.trial150.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53588.log.gz 2026-02-21T23:08:51.831 INFO:teuthology.orchestra.run.trial150.stderr:/var/log/ceph/ceph-client.admin.57375.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56781.log 2026-02-21T23:08:51.831 INFO:teuthology.orchestra.run.trial150.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57375.log.gz 2026-02-21T23:08:51.831 INFO:teuthology.orchestra.run.trial150.stderr:/var/log/ceph/ceph-client.admin.58336.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55071.log 2026-02-21T23:08:51.831 INFO:teuthology.orchestra.run.trial150.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58336.log.gz 2026-02-21T23:08:51.831 INFO:teuthology.orchestra.run.trial150.stderr:/var/log/ceph/ceph-client.admin.56781.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57895.log 2026-02-21T23:08:51.831 INFO:teuthology.orchestra.run.trial150.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56781.log.gz 2026-02-21T23:08:51.832 INFO:teuthology.orchestra.run.trial150.stderr:/var/log/ceph/ceph-client.admin.55071.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56307.log 2026-02-21T23:08:51.832 INFO:teuthology.orchestra.run.trial150.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55071.log.gz 2026-02-21T23:08:51.832 INFO:teuthology.orchestra.run.trial150.stderr:/var/log/ceph/ceph-client.admin.57895.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55072.log 2026-02-21T23:08:51.832 INFO:teuthology.orchestra.run.trial150.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57895.log.gz 2026-02-21T23:08:51.832 INFO:teuthology.orchestra.run.trial150.stderr:/var/log/ceph/ceph-client.admin.56307.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55706.log 2026-02-21T23:08:51.832 INFO:teuthology.orchestra.run.trial150.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56307.log.gz 2026-02-21T23:08:51.832 INFO:teuthology.orchestra.run.trial150.stderr:/var/log/ceph/ceph-client.admin.55072.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55072.log.gz 2026-02-21T23:08:51.832 INFO:teuthology.orchestra.run.trial150.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56756.log 2026-02-21T23:08:51.832 INFO:teuthology.orchestra.run.trial150.stderr:/var/log/ceph/ceph-client.admin.55706.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57796.log 2026-02-21T23:08:51.832 INFO:teuthology.orchestra.run.trial150.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55706.log.gz 2026-02-21T23:08:51.832 INFO:teuthology.orchestra.run.trial150.stderr:/var/log/ceph/ceph-client.admin.56756.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54950.log 2026-02-21T23:08:51.832 INFO:teuthology.orchestra.run.trial150.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56756.log.gz 2026-02-21T23:08:51.833 INFO:teuthology.orchestra.run.trial150.stderr:/var/log/ceph/ceph-client.admin.57796.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53539.log 2026-02-21T23:08:51.833 INFO:teuthology.orchestra.run.trial150.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57796.log.gz 2026-02-21T23:08:51.833 INFO:teuthology.orchestra.run.trial150.stderr:/var/log/ceph/ceph-client.admin.54950.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48926.log 2026-02-21T23:08:51.833 INFO:teuthology.orchestra.run.trial150.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54950.log.gz 2026-02-21T23:08:51.833 INFO:teuthology.orchestra.run.trial150.stderr:/var/log/ceph/ceph-client.admin.53539.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55249.log 2026-02-21T23:08:51.833 INFO:teuthology.orchestra.run.trial150.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53539.log.gz 2026-02-21T23:08:51.833 INFO:teuthology.orchestra.run.trial150.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54172.log 2026-02-21T23:08:51.833 INFO:teuthology.orchestra.run.trial150.stderr:/var/log/ceph/ceph-client.admin.48926.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48926.log.gz 2026-02-21T23:08:51.833 INFO:teuthology.orchestra.run.trial150.stderr:/var/log/ceph/ceph-client.admin.55249.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55233.log 2026-02-21T23:08:51.833 INFO:teuthology.orchestra.run.trial150.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55249.log.gz 2026-02-21T23:08:51.833 INFO:teuthology.orchestra.run.trial150.stderr:/var/log/ceph/ceph-client.admin.54172.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2026-02-21T23:08:51.834 INFO:teuthology.orchestra.run.trial150.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54172.log.gz 2026-02-21T23:08:51.834 INFO:teuthology.orchestra.run.trial150.stderr:/var/log/ceph/ceph-client.admin.55233.log: gzip -5 --verbose 0.0% -- -- replaced with /var/log/ceph/ceph-client.admin.55233.log.gz /var/log/ceph/ceph-client.admin.57230.log 2026-02-21T23:08:51.834 INFO:teuthology.orchestra.run.trial150.stderr: 2026-02-21T23:08:51.834 INFO:teuthology.orchestra.run.trial150.stderr:/var/log/ceph/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58091.log 2026-02-21T23:08:51.834 INFO:teuthology.orchestra.run.trial150.stderr:/var/log/ceph/ceph-client.admin.57230.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54180.log 2026-02-21T23:08:51.834 INFO:teuthology.orchestra.run.trial150.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57230.log.gz 2026-02-21T23:08:51.834 INFO:teuthology.orchestra.run.trial150.stderr:/var/log/ceph/ceph-client.admin.58091.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56606.log 2026-02-21T23:08:51.834 INFO:teuthology.orchestra.run.trial150.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58091.log.gz 2026-02-21T23:08:51.834 INFO:teuthology.orchestra.run.trial150.stderr:/var/log/ceph/ceph-client.admin.54180.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58973.log 2026-02-21T23:08:51.834 INFO:teuthology.orchestra.run.trial150.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54180.log.gz 2026-02-21T23:08:51.834 INFO:teuthology.orchestra.run.trial150.stderr:/var/log/ceph/ceph-client.admin.56606.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56632.log 2026-02-21T23:08:51.835 INFO:teuthology.orchestra.run.trial150.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56606.log.gz 2026-02-21T23:08:51.835 INFO:teuthology.orchestra.run.trial150.stderr:/var/log/ceph/ceph-client.admin.58973.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57649.log 2026-02-21T23:08:51.835 INFO:teuthology.orchestra.run.trial150.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58973.log.gz 2026-02-21T23:08:51.835 INFO:teuthology.orchestra.run.trial150.stderr:/var/log/ceph/ceph-client.admin.56632.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58189.log 2026-02-21T23:08:51.835 INFO:teuthology.orchestra.run.trial150.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56632.log.gz 2026-02-21T23:08:51.835 INFO:teuthology.orchestra.run.trial150.stderr:/var/log/ceph/ceph-client.admin.57649.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57302.log 2026-02-21T23:08:51.835 INFO:teuthology.orchestra.run.trial150.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57649.log.gz 2026-02-21T23:08:51.835 INFO:teuthology.orchestra.run.trial150.stderr:/var/log/ceph/ceph-client.admin.58189.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56956.log 2026-02-21T23:08:51.835 INFO:teuthology.orchestra.run.trial150.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58189.log.gz 2026-02-21T23:08:51.835 INFO:teuthology.orchestra.run.trial150.stderr: 92.4%/var/log/ceph/ceph-client.admin.57302.log: -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2026-02-21T23:08:51.835 INFO:teuthology.orchestra.run.trial150.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56882.log 2026-02-21T23:08:51.835 INFO:teuthology.orchestra.run.trial150.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57302.log.gz 2026-02-21T23:08:51.836 INFO:teuthology.orchestra.run.trial150.stderr:/var/log/ceph/ceph-client.admin.56956.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54460.log 2026-02-21T23:08:51.836 INFO:teuthology.orchestra.run.trial150.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56956.log.gz 2026-02-21T23:08:51.836 INFO:teuthology.orchestra.run.trial150.stderr:/var/log/ceph/ceph-client.admin.56882.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56882.log.gz 2026-02-21T23:08:51.836 INFO:teuthology.orchestra.run.trial150.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56432.log 2026-02-21T23:08:51.836 INFO:teuthology.orchestra.run.trial150.stderr:/var/log/ceph/ceph-client.admin.54460.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54460.log.gz 2026-02-21T23:08:51.836 INFO:teuthology.orchestra.run.trial150.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57474.log 2026-02-21T23:08:51.836 INFO:teuthology.orchestra.run.trial150.stderr:/var/log/ceph/ceph-client.admin.56432.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56432.log.gz 2026-02-21T23:08:51.836 INFO:teuthology.orchestra.run.trial150.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58581.log 2026-02-21T23:08:51.836 INFO:teuthology.orchestra.run.trial150.stderr:/var/log/ceph/ceph-client.admin.57474.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57474.log.gz 2026-02-21T23:08:51.836 INFO:teuthology.orchestra.run.trial150.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54693.log 2026-02-21T23:08:51.836 INFO:teuthology.orchestra.run.trial150.stderr:/var/log/ceph/ceph-client.admin.58581.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58581.log.gz 2026-02-21T23:08:51.836 INFO:teuthology.orchestra.run.trial150.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55234.log 2026-02-21T23:08:51.837 INFO:teuthology.orchestra.run.trial150.stderr:/var/log/ceph/ceph-client.admin.54693.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54693.log.gz 2026-02-21T23:08:51.837 INFO:teuthology.orchestra.run.trial150.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2026-02-21T23:08:51.837 INFO:teuthology.orchestra.run.trial150.stderr:/var/log/ceph/ceph-client.admin.55234.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55234.log.gz 2026-02-21T23:08:51.837 INFO:teuthology.orchestra.run.trial150.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58875.log 2026-02-21T23:08:51.837 INFO:teuthology.orchestra.run.trial150.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48115.log 2026-02-21T23:08:51.837 INFO:teuthology.orchestra.run.trial150.stderr:/var/log/ceph/ceph-client.admin.58875.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58875.log.gz 2026-02-21T23:08:51.837 INFO:teuthology.orchestra.run.trial150.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47625.log 2026-02-21T23:08:51.837 INFO:teuthology.orchestra.run.trial150.stderr:/var/log/ceph/ceph-client.admin.48115.log: 90.1% -- replaced with /var/log/ceph/ceph.audit.log.gz 2026-02-21T23:08:51.837 INFO:teuthology.orchestra.run.trial150.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55250.log 2026-02-21T23:08:51.837 INFO:teuthology.orchestra.run.trial150.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48115.log.gz 2026-02-21T23:08:51.837 INFO:teuthology.orchestra.run.trial150.stderr:/var/log/ceph/ceph-client.admin.47625.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47625.log.gz 2026-02-21T23:08:51.837 INFO:teuthology.orchestra.run.trial150.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57205.log 2026-02-21T23:08:51.837 INFO:teuthology.orchestra.run.trial150.stderr:/var/log/ceph/ceph-client.admin.55250.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55250.log.gz 2026-02-21T23:08:51.837 INFO:teuthology.orchestra.run.trial150.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56981.log 2026-02-21T23:08:51.838 INFO:teuthology.orchestra.run.trial150.stderr:/var/log/ceph/ceph-client.admin.57205.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54699.log 2026-02-21T23:08:51.838 INFO:teuthology.orchestra.run.trial150.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57205.log.gz 2026-02-21T23:08:51.838 INFO:teuthology.orchestra.run.trial150.stderr:/var/log/ceph/ceph-client.admin.56981.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56981.log.gz 2026-02-21T23:08:51.838 INFO:teuthology.orchestra.run.trial150.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48382.log 2026-02-21T23:08:51.838 INFO:teuthology.orchestra.run.trial150.stderr:/var/log/ceph/ceph-client.admin.54699.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54699.log.gz 2026-02-21T23:08:51.838 INFO:teuthology.orchestra.run.trial150.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.6.log 2026-02-21T23:08:51.838 INFO:teuthology.orchestra.run.trial150.stderr:/var/log/ceph/ceph-client.admin.48382.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48382.log.gz 2026-02-21T23:08:51.838 INFO:teuthology.orchestra.run.trial150.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2026-02-21T23:08:51.838 INFO:teuthology.orchestra.run.trial150.stderr:/var/log/ceph/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55560.log 2026-02-21T23:08:51.838 INFO:teuthology.orchestra.run.trial150.stderr:/var/log/ceph/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57698.log 2026-02-21T23:08:51.838 INFO:teuthology.orchestra.run.trial150.stderr:/var/log/ceph/ceph-client.admin.55560.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55560.log.gz 2026-02-21T23:08:51.838 INFO:teuthology.orchestra.run.trial150.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57104.log 2026-02-21T23:08:51.838 INFO:teuthology.orchestra.run.trial150.stderr:/var/log/ceph/ceph-client.admin.57698.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57698.log.gz 2026-02-21T23:08:51.838 INFO:teuthology.orchestra.run.trial150.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59118.log 2026-02-21T23:08:51.839 INFO:teuthology.orchestra.run.trial150.stderr:/var/log/ceph/ceph-client.admin.57104.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57104.log.gz 2026-02-21T23:08:51.839 INFO:teuthology.orchestra.run.trial150.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56582.log 2026-02-21T23:08:51.839 INFO:teuthology.orchestra.run.trial150.stderr:/var/log/ceph/ceph-client.admin.59118.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59118.log.gz 2026-02-21T23:08:51.839 INFO:teuthology.orchestra.run.trial150.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54185.log 2026-02-21T23:08:51.839 INFO:teuthology.orchestra.run.trial150.stderr:/var/log/ceph/ceph-client.admin.56582.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56582.log.gz 2026-02-21T23:08:51.839 INFO:teuthology.orchestra.run.trial150.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55558.log 2026-02-21T23:08:51.839 INFO:teuthology.orchestra.run.trial150.stderr:/var/log/ceph/ceph-client.admin.54185.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54185.log.gz 2026-02-21T23:08:51.839 INFO:teuthology.orchestra.run.trial150.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.37780.log 2026-02-21T23:08:51.839 INFO:teuthology.orchestra.run.trial150.stderr:/var/log/ceph/ceph-client.admin.55558.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55558.log.gz 2026-02-21T23:08:51.839 INFO:teuthology.orchestra.run.trial150.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55726.log 2026-02-21T23:08:51.839 INFO:teuthology.orchestra.run.trial150.stderr:/var/log/ceph/ceph.tmp-client.admin.37780.log: gzip -5 0.0% --verbose -- -- replaced with /var/log/ceph/ceph.tmp-client.admin.37780.log.gz /var/log/ceph/ceph-client.admin.48288.log 2026-02-21T23:08:51.839 INFO:teuthology.orchestra.run.trial150.stderr: 2026-02-21T23:08:51.839 INFO:teuthology.orchestra.run.trial150.stderr:/var/log/ceph/ceph-client.admin.55726.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55726.log.gz 2026-02-21T23:08:51.839 INFO:teuthology.orchestra.run.trial150.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56408.log 2026-02-21T23:08:51.839 INFO:teuthology.orchestra.run.trial150.stderr:/var/log/ceph/ceph-client.admin.48288.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48288.log.gz 2026-02-21T23:08:51.839 INFO:teuthology.orchestra.run.trial150.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53748.log 2026-02-21T23:08:51.839 INFO:teuthology.orchestra.run.trial150.stderr:/var/log/ceph/ceph-client.admin.56408.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56408.log.gz 2026-02-21T23:08:51.840 INFO:teuthology.orchestra.run.trial150.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57154.log 2026-02-21T23:08:51.840 INFO:teuthology.orchestra.run.trial150.stderr:/var/log/ceph/ceph-client.admin.53748.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53748.log.gz 2026-02-21T23:08:51.840 INFO:teuthology.orchestra.run.trial150.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58679.log 2026-02-21T23:08:51.840 INFO:teuthology.orchestra.run.trial150.stderr:/var/log/ceph/ceph-client.admin.57154.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57154.log.gz 2026-02-21T23:08:51.840 INFO:teuthology.orchestra.run.trial150.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54703.log 2026-02-21T23:08:51.840 INFO:teuthology.orchestra.run.trial150.stderr:/var/log/ceph/ceph-client.admin.58679.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58679.log.gz 2026-02-21T23:08:51.840 INFO:teuthology.orchestra.run.trial150.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55235.log 2026-02-21T23:08:51.840 INFO:teuthology.orchestra.run.trial150.stderr:/var/log/ceph/ceph-client.admin.54703.log: gzip -5 --verbose 0.0% -- /var/log/ceph/ceph-client.admin.53895.log -- replaced with /var/log/ceph/ceph-client.admin.54703.log.gz 2026-02-21T23:08:51.840 INFO:teuthology.orchestra.run.trial150.stderr: 2026-02-21T23:08:51.840 INFO:teuthology.orchestra.run.trial150.stderr:/var/log/ceph/ceph-client.admin.55235.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55235.log.gz 2026-02-21T23:08:51.840 INFO:teuthology.orchestra.run.trial150.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56683.log 2026-02-21T23:08:51.840 INFO:teuthology.orchestra.run.trial150.stderr:/var/log/ceph/ceph-client.admin.53895.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53895.log.gz 2026-02-21T23:08:51.840 INFO:teuthology.orchestra.run.trial150.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57450.log 2026-02-21T23:08:51.840 INFO:teuthology.orchestra.run.trial150.stderr:/var/log/ceph/ceph-client.admin.56683.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56683.log.gz 2026-02-21T23:08:51.840 INFO:teuthology.orchestra.run.trial150.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59071.log 2026-02-21T23:08:51.840 INFO:teuthology.orchestra.run.trial150.stderr:/var/log/ceph/ceph-client.admin.57450.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57450.log.gz 2026-02-21T23:08:51.840 INFO:teuthology.orchestra.run.trial150.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53846.log 2026-02-21T23:08:51.840 INFO:teuthology.orchestra.run.trial150.stderr:/var/log/ceph/ceph-client.admin.59071.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54951.log 2026-02-21T23:08:51.840 INFO:teuthology.orchestra.run.trial150.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59071.log.gz 2026-02-21T23:08:51.840 INFO:teuthology.orchestra.run.trial150.stderr:/var/log/ceph/ceph-client.admin.53846.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53846.log.gz 2026-02-21T23:08:51.840 INFO:teuthology.orchestra.run.trial150.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58287.log 2026-02-21T23:08:51.840 INFO:teuthology.orchestra.run.trial150.stderr:/var/log/ceph/ceph-client.admin.54951.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54951.log.gz 2026-02-21T23:08:51.840 INFO:teuthology.orchestra.run.trial150.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58434.log 2026-02-21T23:08:51.840 INFO:teuthology.orchestra.run.trial150.stderr:/var/log/ceph/ceph-client.admin.58287.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58287.log.gz 2026-02-21T23:08:51.840 INFO:teuthology.orchestra.run.trial150.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57425.log 2026-02-21T23:08:51.841 INFO:teuthology.orchestra.run.trial150.stderr: 91.9% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2026-02-21T23:08:51.841 INFO:teuthology.orchestra.run.trial150.stderr:/var/log/ceph/ceph-client.admin.58434.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58434.log.gz 2026-02-21T23:08:51.841 INFO:teuthology.orchestra.run.trial150.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55414.log 2026-02-21T23:08:51.841 INFO:teuthology.orchestra.run.trial150.stderr:/var/log/ceph/ceph-client.admin.57425.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57425.log.gz 2026-02-21T23:08:51.841 INFO:teuthology.orchestra.run.trial150.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2026-02-21T23:08:51.841 INFO:teuthology.orchestra.run.trial150.stderr:/var/log/ceph/ceph-client.admin.55414.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55414.log.gz 2026-02-21T23:08:51.841 INFO:teuthology.orchestra.run.trial150.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54461.log 2026-02-21T23:08:51.841 INFO:teuthology.orchestra.run.trial150.stderr:/var/log/ceph/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57129.log 2026-02-21T23:08:51.841 INFO:teuthology.orchestra.run.trial150.stderr:/var/log/ceph/ceph-client.admin.54461.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54461.log.gz 2026-02-21T23:08:51.841 INFO:teuthology.orchestra.run.trial150.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57178.log 2026-02-21T23:08:51.841 INFO:teuthology.orchestra.run.trial150.stderr:/var/log/ceph/ceph-client.admin.57129.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57129.log.gz 2026-02-21T23:08:51.841 INFO:teuthology.orchestra.run.trial150.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56806.log 2026-02-21T23:08:51.841 INFO:teuthology.orchestra.run.trial150.stderr:/var/log/ceph/ceph-client.admin.57178.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57178.log.gz 2026-02-21T23:08:51.841 INFO:teuthology.orchestra.run.trial150.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57278.log 2026-02-21T23:08:51.841 INFO:teuthology.orchestra.run.trial150.stderr:/var/log/ceph/ceph-client.admin.56806.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56806.log.gz 2026-02-21T23:08:51.841 INFO:teuthology.orchestra.run.trial150.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56331.log 2026-02-21T23:08:51.841 INFO:teuthology.orchestra.run.trial150.stderr:/var/log/ceph/ceph-client.admin.57278.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57278.log.gz 2026-02-21T23:08:51.841 INFO:teuthology.orchestra.run.trial150.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53797.log 2026-02-21T23:08:51.841 INFO:teuthology.orchestra.run.trial150.stderr:/var/log/ceph/ceph-client.admin.56331.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56331.log.gz 2026-02-21T23:08:51.841 INFO:teuthology.orchestra.run.trial150.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58532.log 2026-02-21T23:08:51.841 INFO:teuthology.orchestra.run.trial150.stderr:/var/log/ceph/ceph-client.admin.53797.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53797.log.gz 2026-02-21T23:08:51.841 INFO:teuthology.orchestra.run.trial150.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48335.log 2026-02-21T23:08:51.841 INFO:teuthology.orchestra.run.trial150.stderr:/var/log/ceph/ceph-client.admin.58532.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58532.log.gz 2026-02-21T23:08:51.841 INFO:teuthology.orchestra.run.trial150.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56480.log 2026-02-21T23:08:51.841 INFO:teuthology.orchestra.run.trial150.stderr:/var/log/ceph/ceph-client.admin.48335.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57030.log 2026-02-21T23:08:51.841 INFO:teuthology.orchestra.run.trial150.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48335.log.gz 2026-02-21T23:08:51.841 INFO:teuthology.orchestra.run.trial150.stderr:/var/log/ceph/ceph-client.admin.56480.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56480.log.gz 2026-02-21T23:08:51.842 INFO:teuthology.orchestra.run.trial150.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58630.log 2026-02-21T23:08:51.842 INFO:teuthology.orchestra.run.trial150.stderr:/var/log/ceph/ceph-client.admin.57030.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57030.log.gz 2026-02-21T23:08:51.842 INFO:teuthology.orchestra.run.trial150.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2026-02-21T23:08:51.842 INFO:teuthology.orchestra.run.trial150.stderr:/var/log/ceph/ceph-client.admin.58630.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58630.log.gz 2026-02-21T23:08:51.842 INFO:teuthology.orchestra.run.trial150.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55725.log 2026-02-21T23:08:51.842 INFO:teuthology.orchestra.run.trial150.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.8.log 2026-02-21T23:08:51.842 INFO:teuthology.orchestra.run.trial150.stderr:/var/log/ceph/ceph-client.admin.55725.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55725.log.gz 2026-02-21T23:08:51.842 INFO:teuthology.orchestra.run.trial150.stderr: 85.6% -- replaced with /var/log/ceph/ceph.log.gz 2026-02-21T23:08:51.842 INFO:teuthology.orchestra.run.trial150.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57400.log 2026-02-21T23:08:51.842 INFO:teuthology.orchestra.run.trial150.stderr:/var/log/ceph/ceph-osd.8.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57326.log 2026-02-21T23:08:51.842 INFO:teuthology.orchestra.run.trial150.stderr: 92.1% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2026-02-21T23:08:51.842 INFO:teuthology.orchestra.run.trial150.stderr:/var/log/ceph/ceph-client.admin.57400.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57400.log.gz 2026-02-21T23:08:51.842 INFO:teuthology.orchestra.run.trial150.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48163.log 2026-02-21T23:08:51.842 INFO:teuthology.orchestra.run.trial150.stderr:/var/log/ceph/ceph-client.admin.57326.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57326.log.gz 2026-02-21T23:08:51.842 INFO:teuthology.orchestra.run.trial150.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.9.log 2026-02-21T23:08:51.842 INFO:teuthology.orchestra.run.trial150.stderr:/var/log/ceph/ceph-client.admin.48163.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48163.log.gz 2026-02-21T23:08:51.842 INFO:teuthology.orchestra.run.trial150.stderr:gzip 92.1% -5 -- replaced with /var/log/ceph/ceph-osd.7.log.gz --verbose 2026-02-21T23:08:51.842 INFO:teuthology.orchestra.run.trial150.stderr: -- /var/log/ceph/ceph-client.admin.58238.log 2026-02-21T23:08:51.842 INFO:teuthology.orchestra.run.trial150.stderr:/var/log/ceph/ceph-osd.9.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55953.log 2026-02-21T23:08:51.842 INFO:teuthology.orchestra.run.trial150.stderr:/var/log/ceph/ceph-client.admin.58238.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58238.log.gz 2026-02-21T23:08:51.842 INFO:teuthology.orchestra.run.trial150.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57054.log 2026-02-21T23:08:51.842 INFO:teuthology.orchestra.run.trial150.stderr:/var/log/ceph/ceph-client.admin.55953.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56279.log 2026-02-21T23:08:51.842 INFO:teuthology.orchestra.run.trial150.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55953.log.gz 2026-02-21T23:08:51.842 INFO:teuthology.orchestra.run.trial150.stderr:/var/log/ceph/ceph-client.admin.57054.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57054.log.gz 2026-02-21T23:08:51.842 INFO:teuthology.orchestra.run.trial150.stderr: 92.3% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2026-02-21T23:08:51.842 INFO:teuthology.orchestra.run.trial150.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54717.log 2026-02-21T23:08:51.842 INFO:teuthology.orchestra.run.trial150.stderr:/var/log/ceph/ceph-client.admin.56279.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58042.log 2026-02-21T23:08:51.843 INFO:teuthology.orchestra.run.trial150.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56279.log.gz 2026-02-21T23:08:51.843 INFO:teuthology.orchestra.run.trial150.stderr:/var/log/ceph/ceph-client.admin.54717.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54184.log 2026-02-21T23:08:51.843 INFO:teuthology.orchestra.run.trial150.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54717.log.gz 2026-02-21T23:08:51.843 INFO:teuthology.orchestra.run.trial150.stderr:/var/log/ceph/ceph-client.admin.58042.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58042.log.gz 2026-02-21T23:08:51.843 INFO:teuthology.orchestra.run.trial150.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57993.log 2026-02-21T23:08:51.843 INFO:teuthology.orchestra.run.trial150.stderr:/var/log/ceph/ceph-client.admin.54184.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57944.log 2026-02-21T23:08:51.843 INFO:teuthology.orchestra.run.trial150.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54184.log.gz 2026-02-21T23:08:51.843 INFO:teuthology.orchestra.run.trial150.stderr:/var/log/ceph/ceph-client.admin.57993.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55561.log 2026-02-21T23:08:51.843 INFO:teuthology.orchestra.run.trial150.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57993.log.gz 2026-02-21T23:08:51.843 INFO:teuthology.orchestra.run.trial150.stderr:/var/log/ceph/ceph-client.admin.57944.log: gzip 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57944.log.gz -5 2026-02-21T23:08:51.843 INFO:teuthology.orchestra.run.trial150.stderr: --verbose -- /var/log/ceph/ceph-client.admin.56857.log 2026-02-21T23:08:51.843 INFO:teuthology.orchestra.run.trial150.stderr:/var/log/ceph/ceph-client.admin.55561.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55561.log.gz 2026-02-21T23:08:51.843 INFO:teuthology.orchestra.run.trial150.stderr:/var/log/ceph/ceph-client.admin.56857.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56857.log.gz 2026-02-21T23:08:51.843 INFO:teuthology.orchestra.run.trial150.stderr: 91.5% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2026-02-21T23:08:51.844 INFO:teuthology.orchestra.run.trial150.stderr: 92.2% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2026-02-21T23:08:51.844 INFO:teuthology.orchestra.run.trial150.stderr: 92.2% -- replaced with /var/log/ceph/ceph-osd.6.log.gz 2026-02-21T23:08:51.844 INFO:teuthology.orchestra.run.trial150.stderr: 92.3% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2026-02-21T23:08:51.844 INFO:teuthology.orchestra.run.trial150.stderr: 92.3% -- replaced with /var/log/ceph/ceph-osd.8.log.gz 2026-02-21T23:08:51.844 INFO:teuthology.orchestra.run.trial150.stderr: 92.1% -- replaced with /var/log/ceph/ceph-osd.9.log.gz 2026-02-21T23:08:51.863 INFO:teuthology.orchestra.run.trial150.stderr: 88.5% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2026-02-21T23:08:51.864 INFO:teuthology.orchestra.run.trial150.stderr: 2026-02-21T23:08:51.864 INFO:teuthology.orchestra.run.trial150.stderr:real 0m0.095s 2026-02-21T23:08:51.864 INFO:teuthology.orchestra.run.trial150.stderr:user 0m0.392s 2026-02-21T23:08:51.864 INFO:teuthology.orchestra.run.trial150.stderr:sys 0m0.076s 2026-02-21T23:08:51.865 INFO:tasks.ceph:Archiving logs... 2026-02-21T23:08:51.865 DEBUG:teuthology.misc:Transferring archived files from trial150:/var/log/ceph to /home/teuthworker/mnt/teuthology/nmordech-2026-02-21_21:55:34-rados-wip-rocky10-branch-of-the-day-2026-02-21-1771692874-tentacle-distro-default-trial/63256/remote/trial150/log 2026-02-21T23:08:51.866 DEBUG:teuthology.orchestra.run.trial150:> sudo tar c -f - -C /var/log/ceph -- . 2026-02-21T23:08:52.095 DEBUG:teuthology.run_tasks:Unwinding manager install 2026-02-21T23:08:52.111 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2026-02-21T23:08:52.112 DEBUG:teuthology.orchestra.run.trial150:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2026-02-21T23:08:52.222 DEBUG:teuthology.run_tasks:Unwinding manager clock 2026-02-21T23:08:52.233 INFO:teuthology.task.clock:Checking final clock skew... 2026-02-21T23:08:52.233 DEBUG:teuthology.orchestra.run.trial150:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2026-02-21T23:08:52.249 INFO:teuthology.orchestra.run.trial150.stderr:bash: line 1: ntpq: command not found 2026-02-21T23:08:52.271 INFO:teuthology.orchestra.run.trial150.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2026-02-21T23:08:52.271 INFO:teuthology.orchestra.run.trial150.stdout:=============================================================================== 2026-02-21T23:08:52.271 INFO:teuthology.orchestra.run.trial150.stdout:^* 149.248.12.167.vultruser> 3 6 177 21 +223us[ -40us] +/- 36ms 2026-02-21T23:08:52.271 INFO:teuthology.orchestra.run.trial150.stdout:^+ 139-177-202-26.ip.linode> 2 6 177 22 -6790us[-7053us] +/- 79ms 2026-02-21T23:08:52.271 INFO:teuthology.orchestra.run.trial150.stdout:^+ x.ns.gin.ntt.net 2 6 177 86 +3128us[ +299us] +/- 46ms 2026-02-21T23:08:52.271 INFO:teuthology.orchestra.run.trial150.stdout:^+ sushi.ruselabs.com 2 6 177 20 -658us[ -658us] +/- 39ms 2026-02-21T23:08:52.271 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2026-02-21T23:08:52.283 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2026-02-21T23:08:52.283 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2026-02-21T23:08:52.298 DEBUG:teuthology.orchestra.run.trial150:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2026-02-21T23:08:52.366 DEBUG:teuthology.orchestra.run.trial150:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2026-02-21T23:08:52.392 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-21T23:08:52.393 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2026-02-21T23:08:52.409 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2026-02-21T23:08:52.424 INFO:teuthology.task.internal:Duration was 382.148678 seconds 2026-02-21T23:08:52.425 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2026-02-21T23:08:52.439 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2026-02-21T23:08:52.440 DEBUG:teuthology.orchestra.run.trial150:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2026-02-21T23:08:52.477 INFO:teuthology.orchestra.run.trial150.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2026-02-21T23:08:52.761 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2026-02-21T23:08:52.761 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@trial150.front.sepia.ceph.com 2026-02-21T23:08:52.761 DEBUG:teuthology.orchestra.run.trial150:> grep -E --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | grep -E -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | grep -E -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2026-02-21T23:08:52.819 INFO:teuthology.task.internal.syslog:Gathering journactl... 2026-02-21T23:08:52.820 DEBUG:teuthology.orchestra.run.trial150:> sudo journalctl > /home/ubuntu/cephtest/archive/syslog/journalctl.log 2026-02-21T23:08:52.922 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2026-02-21T23:08:52.923 DEBUG:teuthology.orchestra.run.trial150:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --max-args=1 --max-procs=0 --verbose --no-run-if-empty -- gzip -5 --verbose -- 2026-02-21T23:08:52.948 INFO:teuthology.orchestra.run.trial150.stderr:gzip -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/misc.log 2026-02-21T23:08:52.948 INFO:teuthology.orchestra.run.trial150.stderr:gzip -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/kern.log 2026-02-21T23:08:52.948 INFO:teuthology.orchestra.run.trial150.stderr:/home/ubuntu/cephtest/archive/syslog/misc.log: gzip 0.0% -5 --verbose -- replaced with /home/ubuntu/cephtest/archive/syslog/misc.log.gz -- 2026-02-21T23:08:52.949 INFO:teuthology.orchestra.run.trial150.stderr: /home/ubuntu/cephtest/archive/syslog/journalctl.log 2026-02-21T23:08:52.949 INFO:teuthology.orchestra.run.trial150.stderr:/home/ubuntu/cephtest/archive/syslog/kern.log: 0.0% -- replaced with /home/ubuntu/cephtest/archive/syslog/kern.log.gz 2026-02-21T23:08:52.958 INFO:teuthology.orchestra.run.trial150.stderr:/home/ubuntu/cephtest/archive/syslog/journalctl.log: 82.1% -- replaced with /home/ubuntu/cephtest/archive/syslog/journalctl.log.gz 2026-02-21T23:08:52.960 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2026-02-21T23:08:52.981 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2026-02-21T23:08:52.981 DEBUG:teuthology.orchestra.run.trial150:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2026-02-21T23:08:53.023 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2026-02-21T23:08:53.043 DEBUG:teuthology.orchestra.run.trial150:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2026-02-21T23:08:53.087 INFO:teuthology.orchestra.run.trial150.stdout:kernel.core_pattern = core 2026-02-21T23:08:53.117 DEBUG:teuthology.orchestra.run.trial150:> test -e /home/ubuntu/cephtest/archive/coredump 2026-02-21T23:08:53.173 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-21T23:08:53.174 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2026-02-21T23:08:53.190 INFO:teuthology.task.internal:Transferring archived files... 2026-02-21T23:08:53.191 DEBUG:teuthology.misc:Transferring archived files from trial150:/home/ubuntu/cephtest/archive to /home/teuthworker/mnt/teuthology/nmordech-2026-02-21_21:55:34-rados-wip-rocky10-branch-of-the-day-2026-02-21-1771692874-tentacle-distro-default-trial/63256/remote/trial150 2026-02-21T23:08:53.191 DEBUG:teuthology.orchestra.run.trial150:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2026-02-21T23:08:53.248 INFO:teuthology.task.internal:Removing archive directory... 2026-02-21T23:08:53.249 DEBUG:teuthology.orchestra.run.trial150:> rm -rf -- /home/ubuntu/cephtest/archive 2026-02-21T23:08:53.297 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2026-02-21T23:08:53.319 INFO:teuthology.task.internal:Not uploading archives. 2026-02-21T23:08:53.319 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2026-02-21T23:08:53.335 INFO:teuthology.task.internal:Tidying up after the test... 2026-02-21T23:08:53.335 DEBUG:teuthology.orchestra.run.trial150:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2026-02-21T23:08:53.354 INFO:teuthology.orchestra.run.trial150.stdout: 917576 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Feb 21 23:08 /home/ubuntu/cephtest 2026-02-21T23:08:53.355 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2026-02-21T23:08:53.393 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2026-02-21T23:08:53.433 INFO:teuthology.run:Summary data: description: rados/singleton-nomsgr/{all/health-warnings mon_election/connectivity rados supported-random-distro$/{centos_latest}} duration: 382.14867782592773 flavor: default owner: scheduled_nmordech@soko04.front.sepia.ceph.com success: true 2026-02-21T23:08:53.433 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com 2026-02-21T23:08:53.551 INFO:teuthology.run:pass