2024-09-16T10:52:00.455 INFO:root:teuthology version: 1.2.2.dev2+g3752d38 2024-09-16T10:52:00.470 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-16T10:52:00.555 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/sangadi-2024-09-16_10:44:09-rbd-wip-bharath9-testing-2024-09-13-1444-quincy-distro-default-smithi/7907146 branch: wip-bharath9-testing-2024-09-13-1444-quincy description: rbd/valgrind/{base/install centos_latest clusters/{fixed-1 openstack} conf/{disable-pool-app} objectstore/bluestore-hybrid validator/memcheck workloads/fsx} email: sangadi@redhat.com first_in_suite: false job_id: '7907146' kernel: kdb: 1 sha1: distro last_in_suite: false machine_type: smithi name: sangadi-2024-09-16_10:44:09-rbd-wip-bharath9-testing-2024-09-13-1444-quincy-distro-default-smithi no_nested_subset: false openstack: - volumes: count: 3 size: 30 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-bharath9-testing-2024-09-13-1444-quincy ceph: conf: global: mon warn on pool no app: false mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 osd: bluefs allocator: hybrid bluestore allocator: hybrid bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd shutdown pgref assert: true flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) sha1: d75f23820806da622f561ee5ee7551015285993f ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log global: osd crush chooseleaf type: 0 osd pool default pg num: 128 osd pool default pgp num: 128 osd pool default size: 2 mon: {} osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: debuginfo: true enable_coprs: - ceph/el9 flavor: default sha1: d75f23820806da622f561ee5ee7551015285993f rbd_fsx: valgrind: - --tool=memcheck thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-bharath9-testing-2024-09-13-1444-quincy env: VALGRIND: --tool=memcheck --leak-check=full sha1: d75f23820806da622f561ee5ee7551015285993f owner: scheduled_sangadi@teuthology priority: 130 repo: https://github.com/ceph/ceph-ci.git roles: - - mon.a - mgr.x - osd.0 - osd.1 - osd.2 - client.0 seed: 9764 sha1: d75f23820806da622f561ee5ee7551015285993f sleep_before_teardown: 0 subset: 111/120000 suite: rbd suite_branch: wip-bharath9-testing-2024-09-13-1444-quincy suite_path: /home/teuthworker/src/github.com_ceph_ceph-c_d75f23820806da622f561ee5ee7551015285993f/qa suite_relpath: qa suite_repo: https://github.com/ceph/ceph-ci.git suite_sha1: d75f23820806da622f561ee5ee7551015285993f targets: smithi102.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBM6iLK7rxqNnDZpwl7wO2SLuZFIQ92XBDe4yi74JjdzO70rvQqlPbil+kMjvlZVerdZ1ptWjMCq0rm2MQlJsDj8= tasks: - install: null - ceph: null - rbd_fsx: clients: - client.0 size: 134217728 teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: 3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a timestamp: 2024-09-16_10:44:09 tube: smithi user: sangadi verbose: true worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856 2024-09-16T10:52:00.555 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_ceph_ceph-c_d75f23820806da622f561ee5ee7551015285993f/qa; will attempt to use it 2024-09-16T10:52:00.556 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_ceph_ceph-c_d75f23820806da622f561ee5ee7551015285993f/qa/tasks 2024-09-16T10:52:00.556 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-09-16T10:52:00.558 INFO:teuthology.task.internal:Checking packages... 2024-09-16T10:52:00.578 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash 'd75f23820806da622f561ee5ee7551015285993f' 2024-09-16T10:52:00.579 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-16T10:52:00.579 INFO:teuthology.packaging:ref: None 2024-09-16T10:52:00.579 INFO:teuthology.packaging:tag: None 2024-09-16T10:52:00.579 INFO:teuthology.packaging:branch: wip-bharath9-testing-2024-09-13-1444-quincy 2024-09-16T10:52:00.579 INFO:teuthology.packaging:sha1: d75f23820806da622f561ee5ee7551015285993f 2024-09-16T10:52:00.579 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-bharath9-testing-2024-09-13-1444-quincy 2024-09-16T10:52:00.733 INFO:teuthology.task.internal:Found packages for ceph version 17.2.7-1643.gd75f2382 2024-09-16T10:52:00.735 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-09-16T10:52:00.742 INFO:teuthology.task.internal:no buildpackages task found 2024-09-16T10:52:00.742 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-09-16T10:52:00.776 INFO:teuthology.task.internal:Saving configuration 2024-09-16T10:52:00.787 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-09-16T10:52:00.800 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-09-16T10:52:00.825 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi102.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/sangadi-2024-09-16_10:44:09-rbd-wip-bharath9-testing-2024-09-13-1444-quincy-distro-default-smithi/7907146', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-16 10:47:13.020954', 'locked_by': 'scheduled_sangadi@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBM6iLK7rxqNnDZpwl7wO2SLuZFIQ92XBDe4yi74JjdzO70rvQqlPbil+kMjvlZVerdZ1ptWjMCq0rm2MQlJsDj8='} 2024-09-16T10:52:00.825 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-09-16T10:52:00.831 INFO:teuthology.task.internal:roles: ubuntu@smithi102.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'osd.0', 'osd.1', 'osd.2', 'client.0'] 2024-09-16T10:52:00.831 INFO:teuthology.run_tasks:Running task console_log... 2024-09-16T10:52:00.885 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f54016b7ac0>, signals=[15]) 2024-09-16T10:52:00.886 INFO:teuthology.run_tasks:Running task internal.connect... 2024-09-16T10:52:00.892 INFO:teuthology.task.internal:Opening connections... 2024-09-16T10:52:00.892 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi102.front.sepia.ceph.com 2024-09-16T10:52:00.893 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi102.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-16T10:52:00.971 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-09-16T10:52:00.977 DEBUG:teuthology.orchestra.run.smithi102:> uname -m 2024-09-16T10:52:01.066 INFO:teuthology.orchestra.run.smithi102.stdout:x86_64 2024-09-16T10:52:01.066 DEBUG:teuthology.orchestra.run.smithi102:> cat /etc/os-release 2024-09-16T10:52:01.123 INFO:teuthology.orchestra.run.smithi102.stdout:NAME="CentOS Stream" 2024-09-16T10:52:01.123 INFO:teuthology.orchestra.run.smithi102.stdout:VERSION="9" 2024-09-16T10:52:01.123 INFO:teuthology.orchestra.run.smithi102.stdout:ID="centos" 2024-09-16T10:52:01.123 INFO:teuthology.orchestra.run.smithi102.stdout:ID_LIKE="rhel fedora" 2024-09-16T10:52:01.124 INFO:teuthology.orchestra.run.smithi102.stdout:VERSION_ID="9" 2024-09-16T10:52:01.124 INFO:teuthology.orchestra.run.smithi102.stdout:PLATFORM_ID="platform:el9" 2024-09-16T10:52:01.124 INFO:teuthology.orchestra.run.smithi102.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-16T10:52:01.124 INFO:teuthology.orchestra.run.smithi102.stdout:ANSI_COLOR="0;31" 2024-09-16T10:52:01.124 INFO:teuthology.orchestra.run.smithi102.stdout:LOGO="fedora-logo-icon" 2024-09-16T10:52:01.124 INFO:teuthology.orchestra.run.smithi102.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-16T10:52:01.124 INFO:teuthology.orchestra.run.smithi102.stdout:HOME_URL="https://centos.org/" 2024-09-16T10:52:01.124 INFO:teuthology.orchestra.run.smithi102.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-16T10:52:01.124 INFO:teuthology.orchestra.run.smithi102.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-16T10:52:01.124 INFO:teuthology.orchestra.run.smithi102.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-16T10:52:01.125 INFO:teuthology.lock.ops:Updating smithi102.front.sepia.ceph.com on lock server 2024-09-16T10:52:01.149 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-09-16T10:52:01.158 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-09-16T10:52:01.200 INFO:teuthology.task.internal:Checking for old test directory... 2024-09-16T10:52:01.200 DEBUG:teuthology.orchestra.run.smithi102:> test '!' -e /home/ubuntu/cephtest 2024-09-16T10:52:01.216 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-09-16T10:52:01.224 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-09-16T10:52:01.224 DEBUG:teuthology.orchestra.run.smithi102:> test -z $(ls -A /var/lib/ceph) 2024-09-16T10:52:01.280 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-09-16T10:52:01.314 INFO:teuthology.run_tasks:Running task kernel... 2024-09-16T10:52:01.330 INFO:teuthology.task.kernel:normalize config orig: {'kdb': 1, 'sha1': 'distro'} 2024-09-16T10:52:01.330 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': 1, 'sha1': 'distro'}}, timeout 300 2024-09-16T10:52:01.330 DEBUG:teuthology.orchestra.run.smithi102:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-16T10:52:01.349 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T10:52:01.350 DEBUG:teuthology.orchestra.run.smithi102:> uname -r 2024-09-16T10:52:01.406 INFO:teuthology.orchestra.run.smithi102.stdout:5.14.0-503.el9.x86_64 2024-09-16T10:52:01.406 INFO:teuthology.task.kernel:Running kernel on smithi102: 5.14.0-503.el9.x86_64 2024-09-16T10:52:01.406 DEBUG:teuthology.orchestra.run.smithi102:> sudo yum install -y kernel 2024-09-16T10:52:03.706 INFO:teuthology.orchestra.run.smithi102.stdout:CentOS Stream 9 - BaseOS 15 MB/s | 8.2 MB 00:00 2024-09-16T10:52:06.182 INFO:teuthology.orchestra.run.smithi102.stdout:CentOS Stream 9 - AppStream 15 MB/s | 20 MB 00:01 2024-09-16T10:52:10.491 INFO:teuthology.orchestra.run.smithi102.stdout:CentOS Stream 9 - CRB 14 MB/s | 6.5 MB 00:00 2024-09-16T10:52:12.150 INFO:teuthology.orchestra.run.smithi102.stdout:CentOS Stream 9 - Extras packages 64 kB/s | 19 kB 00:00 2024-09-16T10:52:13.296 INFO:teuthology.orchestra.run.smithi102.stdout:Extra Packages for Enterprise Linux 22 MB/s | 23 MB 00:01 2024-09-16T10:52:18.827 INFO:teuthology.orchestra.run.smithi102.stdout:lab-extras 16 kB/s | 1.7 kB 00:00 2024-09-16T10:52:20.833 INFO:teuthology.orchestra.run.smithi102.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-16T10:52:20.833 INFO:teuthology.orchestra.run.smithi102.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-16T10:52:20.923 INFO:teuthology.orchestra.run.smithi102.stdout:Dependencies resolved. 2024-09-16T10:52:20.930 INFO:teuthology.orchestra.run.smithi102.stdout:================================================================================ 2024-09-16T10:52:20.930 INFO:teuthology.orchestra.run.smithi102.stdout: Package Architecture Version Repository Size 2024-09-16T10:52:20.930 INFO:teuthology.orchestra.run.smithi102.stdout:================================================================================ 2024-09-16T10:52:20.930 INFO:teuthology.orchestra.run.smithi102.stdout:Installing: 2024-09-16T10:52:20.930 INFO:teuthology.orchestra.run.smithi102.stdout: kernel x86_64 5.14.0-505.el9 baseos 23 k 2024-09-16T10:52:20.930 INFO:teuthology.orchestra.run.smithi102.stdout:Installing dependencies: 2024-09-16T10:52:20.931 INFO:teuthology.orchestra.run.smithi102.stdout: kernel-core x86_64 5.14.0-505.el9 baseos 16 M 2024-09-16T10:52:20.931 INFO:teuthology.orchestra.run.smithi102.stdout: kernel-modules x86_64 5.14.0-505.el9 baseos 35 M 2024-09-16T10:52:20.931 INFO:teuthology.orchestra.run.smithi102.stdout: kernel-modules-core x86_64 5.14.0-505.el9 baseos 29 M 2024-09-16T10:52:20.931 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-16T10:52:20.931 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction Summary 2024-09-16T10:52:20.931 INFO:teuthology.orchestra.run.smithi102.stdout:================================================================================ 2024-09-16T10:52:20.931 INFO:teuthology.orchestra.run.smithi102.stdout:Install 4 Packages 2024-09-16T10:52:20.931 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-16T10:52:20.932 INFO:teuthology.orchestra.run.smithi102.stdout:Total download size: 79 M 2024-09-16T10:52:20.932 INFO:teuthology.orchestra.run.smithi102.stdout:Installed size: 126 M 2024-09-16T10:52:20.932 INFO:teuthology.orchestra.run.smithi102.stdout:Downloading Packages: 2024-09-16T10:52:21.070 INFO:teuthology.orchestra.run.smithi102.stdout:(1/4): kernel-5.14.0-505.el9.x86_64.rpm 261 kB/s | 23 kB 00:00 2024-09-16T10:52:21.421 INFO:teuthology.orchestra.run.smithi102.stdout:(2/4): kernel-core-5.14.0-505.el9.x86_64.rpm 36 MB/s | 16 MB 00:00 2024-09-16T10:52:21.821 INFO:teuthology.orchestra.run.smithi102.stdout:(3/4): kernel-modules-5.14.0-505.el9.x86_64.rpm 41 MB/s | 35 MB 00:00 2024-09-16T10:52:22.246 INFO:teuthology.orchestra.run.smithi102.stdout:(4/4): kernel-modules-core-5.14.0-505.el9.x86_6 24 MB/s | 29 MB 00:01 2024-09-16T10:52:22.246 INFO:teuthology.orchestra.run.smithi102.stdout:-------------------------------------------------------------------------------- 2024-09-16T10:52:22.246 INFO:teuthology.orchestra.run.smithi102.stdout:Total 60 MB/s | 79 MB 00:01 2024-09-16T10:52:22.919 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction check 2024-09-16T10:52:23.274 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction check succeeded. 2024-09-16T10:52:23.274 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction test 2024-09-16T10:52:24.222 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction test succeeded. 2024-09-16T10:52:24.222 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction 2024-09-16T10:52:26.650 INFO:teuthology.orchestra.run.smithi102.stdout: Preparing : 1/1 2024-09-16T10:52:27.337 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : kernel-modules-core-5.14.0-505.el9.x86_64 1/4 2024-09-16T10:52:27.422 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-16T10:52:30.202 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-16T10:52:30.335 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-16T10:52:35.154 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-16T10:52:35.277 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : kernel-5.14.0-505.el9.x86_64 4/4 2024-09-16T10:52:40.023 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-16T10:53:18.368 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 4/4 2024-09-16T10:53:18.413 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 4/4 2024-09-16T10:53:18.772 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: kernel-5.14.0-505.el9.x86_64 4/4 2024-09-16T10:53:18.772 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : kernel-5.14.0-505.el9.x86_64 1/4 2024-09-16T10:53:18.772 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-16T10:53:18.772 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-16T10:53:19.587 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-16T10:53:19.587 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-16T10:53:19.587 INFO:teuthology.orchestra.run.smithi102.stdout:Installed: 2024-09-16T10:53:19.587 INFO:teuthology.orchestra.run.smithi102.stdout: kernel-5.14.0-505.el9.x86_64 2024-09-16T10:53:19.587 INFO:teuthology.orchestra.run.smithi102.stdout: kernel-core-5.14.0-505.el9.x86_64 2024-09-16T10:53:19.588 INFO:teuthology.orchestra.run.smithi102.stdout: kernel-modules-5.14.0-505.el9.x86_64 2024-09-16T10:53:19.588 INFO:teuthology.orchestra.run.smithi102.stdout: kernel-modules-core-5.14.0-505.el9.x86_64 2024-09-16T10:53:19.588 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-16T10:53:19.588 INFO:teuthology.orchestra.run.smithi102.stdout:Complete! 2024-09-16T10:53:19.929 DEBUG:teuthology.orchestra.run.smithi102:> rpm -q kernel | sort -rV | head -n 1 2024-09-16T10:53:20.043 INFO:teuthology.orchestra.run.smithi102.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-16T10:53:20.044 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-16T10:53:20.044 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-503.el9.x86_64 Expected: 5.14.0-505.el9.x86_64 2024-09-16T10:53:20.044 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-16T10:53:20.044 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-16T10:53:20.044 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2024-09-16T10:53:20.044 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi102.front.sepia.ceph.com, path=None, version=distro) 2024-09-16T10:53:20.044 DEBUG:teuthology.orchestra.run.smithi102:> rpm -q kernel | sort -rV | head -n 1 2024-09-16T10:53:20.102 INFO:teuthology.orchestra.run.smithi102.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-16T10:53:20.102 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-16T10:53:20.102 DEBUG:teuthology.orchestra.run.smithi102:> sudo rpm -qi grub2-tools 2024-09-16T10:53:20.134 INFO:teuthology.orchestra.run.smithi102.stdout:Name : grub2-tools 2024-09-16T10:53:20.134 INFO:teuthology.orchestra.run.smithi102.stdout:Epoch : 1 2024-09-16T10:53:20.135 INFO:teuthology.orchestra.run.smithi102.stdout:Version : 2.06 2024-09-16T10:53:20.135 INFO:teuthology.orchestra.run.smithi102.stdout:Release : 82.el9 2024-09-16T10:53:20.135 INFO:teuthology.orchestra.run.smithi102.stdout:Architecture: x86_64 2024-09-16T10:53:20.135 INFO:teuthology.orchestra.run.smithi102.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-16T10:53:20.135 INFO:teuthology.orchestra.run.smithi102.stdout:Group : Unspecified 2024-09-16T10:53:20.135 INFO:teuthology.orchestra.run.smithi102.stdout:Size : 8274670 2024-09-16T10:53:20.135 INFO:teuthology.orchestra.run.smithi102.stdout:License : GPLv3+ 2024-09-16T10:53:20.135 INFO:teuthology.orchestra.run.smithi102.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-16T10:53:20.135 INFO:teuthology.orchestra.run.smithi102.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-16T10:53:20.136 INFO:teuthology.orchestra.run.smithi102.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-16T10:53:20.136 INFO:teuthology.orchestra.run.smithi102.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-16T10:53:20.136 INFO:teuthology.orchestra.run.smithi102.stdout:Packager : builder@centos.org 2024-09-16T10:53:20.136 INFO:teuthology.orchestra.run.smithi102.stdout:Vendor : CentOS 2024-09-16T10:53:20.136 INFO:teuthology.orchestra.run.smithi102.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-16T10:53:20.136 INFO:teuthology.orchestra.run.smithi102.stdout:Summary : Support tools for GRUB. 2024-09-16T10:53:20.136 INFO:teuthology.orchestra.run.smithi102.stdout:Description : 2024-09-16T10:53:20.136 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-16T10:53:20.136 INFO:teuthology.orchestra.run.smithi102.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-16T10:53:20.136 INFO:teuthology.orchestra.run.smithi102.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-16T10:53:20.136 INFO:teuthology.orchestra.run.smithi102.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-16T10:53:20.136 INFO:teuthology.orchestra.run.smithi102.stdout:hardware devices. 2024-09-16T10:53:20.137 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-16T10:53:20.137 INFO:teuthology.orchestra.run.smithi102.stdout:This subpackage provides tools for support of all platforms. 2024-09-16T10:53:20.137 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-16T10:53:20.137 INFO:teuthology.task.kernel:Updating grub on smithi102 to boot 5.14.0-505.el9.x86_64 2024-09-16T10:53:20.138 DEBUG:teuthology.orchestra.run.smithi102:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-16T10:53:20.803 INFO:teuthology.orchestra.run.smithi102.stderr:Generating grub configuration file ... 2024-09-16T10:53:22.065 INFO:teuthology.orchestra.run.smithi102.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-16T10:53:22.094 INFO:teuthology.orchestra.run.smithi102.stderr:done 2024-09-16T10:53:22.096 DEBUG:teuthology.orchestra.run.smithi102:> mktemp 2024-09-16T10:53:22.112 INFO:teuthology.orchestra.run.smithi102.stdout:/tmp/tmp.AbzKGz0sqr 2024-09-16T10:53:22.112 DEBUG:teuthology.orchestra.run.smithi102:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.AbzKGz0sqr 2024-09-16T10:53:22.179 DEBUG:teuthology.orchestra.run.smithi102:> sudo chmod 0666 /tmp/tmp.AbzKGz0sqr 2024-09-16T10:53:22.314 DEBUG:teuthology.orchestra.remote:smithi102:/tmp/tmp.AbzKGz0sqr is 6KB 2024-09-16T10:53:22.363 DEBUG:teuthology.orchestra.run.smithi102:> rm -fr /tmp/tmp.AbzKGz0sqr 2024-09-16T10:53:22.378 DEBUG:teuthology.orchestra.run.smithi102:> sudo /bin/ls /boot/loader/entries || true 2024-09-16T10:53:22.440 INFO:teuthology.orchestra.run.smithi102.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-16T10:53:22.440 INFO:teuthology.orchestra.run.smithi102.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-16T10:53:22.441 INFO:teuthology.orchestra.run.smithi102.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-16T10:53:22.441 INFO:teuthology.orchestra.run.smithi102.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64.conf 2024-09-16T10:53:22.442 DEBUG:teuthology.orchestra.run.smithi102:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64 2024-09-16T10:53:22.620 DEBUG:teuthology.orchestra.run.smithi102:> sudo shutdown -r now 2024-09-16T10:53:52.653 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-09-16T10:53:52.653 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi102.front.sepia.ceph.com' 2024-09-16T10:53:52.654 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi102.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-16T10:54:11.005 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.102 2024-09-16T10:54:20.013 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi102.front.sepia.ceph.com' 2024-09-16T10:54:20.013 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi102.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-16T10:54:23.069 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.102 2024-09-16T10:54:35.081 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi102.front.sepia.ceph.com' 2024-09-16T10:54:35.081 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi102.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-16T10:55:06.923 DEBUG:teuthology.orchestra.run.smithi102:> true 2024-09-16T10:55:07.657 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi102.front.sepia.ceph.com' 2024-09-16T10:55:07.657 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.14.0-505.el9.x86_64"... 2024-09-16T10:55:07.657 DEBUG:teuthology.orchestra.run.smithi102:> uname -r 2024-09-16T10:55:07.672 INFO:teuthology.orchestra.run.smithi102.stdout:5.14.0-505.el9.x86_64 2024-09-16T10:55:07.673 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-505.el9.x86_64 vs 5.14.0-505.el9.x86_64 2024-09-16T10:55:07.673 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-16T10:55:07.673 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-16T10:55:08.674 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-09-16T10:55:08.674 DEBUG:teuthology.orchestra.run.smithi102:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-16T10:55:08.736 INFO:teuthology.orchestra.run.smithi102.stdout:ttyS1 2024-09-16T10:55:08.764 DEBUG:teuthology.parallel:result is None 2024-09-16T10:55:08.764 INFO:teuthology.run_tasks:Running task internal.base... 2024-09-16T10:55:08.771 INFO:teuthology.task.internal:Creating test directory... 2024-09-16T10:55:08.772 DEBUG:teuthology.orchestra.run.smithi102:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-16T10:55:08.789 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-09-16T10:55:08.815 INFO:teuthology.run_tasks:Running task internal.archive... 2024-09-16T10:55:08.849 INFO:teuthology.task.internal:Creating archive directory... 2024-09-16T10:55:08.849 DEBUG:teuthology.orchestra.run.smithi102:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-16T10:55:08.905 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-09-16T10:55:08.911 INFO:teuthology.task.internal:Enabling coredump saving... 2024-09-16T10:55:08.911 DEBUG:teuthology.orchestra.run.smithi102:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-16T10:55:08.948 INFO:teuthology.orchestra.run.smithi102.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-16T10:55:08.958 INFO:teuthology.orchestra.run.smithi102.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-16T10:55:08.960 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-09-16T10:55:08.968 INFO:teuthology.task.internal:Configuring sudo... 2024-09-16T10:55:08.968 DEBUG:teuthology.orchestra.run.smithi102:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-16T10:55:09.026 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-09-16T10:55:09.035 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-09-16T10:55:09.035 DEBUG:teuthology.orchestra.run.smithi102:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-16T10:55:09.082 DEBUG:teuthology.orchestra.run.smithi102:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-16T10:55:09.166 DEBUG:teuthology.orchestra.run.smithi102:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-16T10:55:09.211 DEBUG:teuthology.orchestra.run.smithi102:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-16T10:55:09.287 DEBUG:teuthology.orchestra.run.smithi102:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-16T10:55:09.333 DEBUG:teuthology.orchestra.run.smithi102:> set -ex 2024-09-16T10:55:09.333 DEBUG:teuthology.orchestra.run.smithi102:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-16T10:55:09.398 DEBUG:teuthology.orchestra.run.smithi102:> sudo service rsyslog restart 2024-09-16T10:55:09.477 INFO:teuthology.orchestra.run.smithi102.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-16T10:55:09.859 INFO:teuthology.run_tasks:Running task internal.timer... 2024-09-16T10:55:09.867 INFO:teuthology.task.internal:Starting timer... 2024-09-16T10:55:09.867 INFO:teuthology.run_tasks:Running task pcp... 2024-09-16T10:55:09.911 INFO:teuthology.run_tasks:Running task selinux... 2024-09-16T10:55:09.943 DEBUG:teuthology.orchestra.run.smithi102:> sudo service auditd rotate 2024-09-16T10:55:10.004 INFO:teuthology.orchestra.run.smithi102.stdout:Rotating logs: 2024-09-16T10:55:10.006 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-09-16T10:55:10.006 DEBUG:teuthology.orchestra.run.smithi102:> /usr/sbin/getenforce 2024-09-16T10:55:10.062 INFO:teuthology.orchestra.run.smithi102.stdout:Permissive 2024-09-16T10:55:10.062 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi102.front.sepia.ceph.com': 'permissive'} 2024-09-16T10:55:10.062 DEBUG:teuthology.orchestra.run.smithi102:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2024-09-16T10:55:10.087 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T10:55:10.087 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-09-16T10:55:10.088 DEBUG:teuthology.orchestra.run.smithi102:> sudo /usr/sbin/setenforce permissive 2024-09-16T10:55:10.169 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-09-16T10:55:10.178 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-09-16T10:55:10.190 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-09-16T10:55:10.190 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi102.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-09-16T10:58:50.413 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi102.front.sepia.ceph.com')] 2024-09-16T10:58:50.414 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi102.front.sepia.ceph.com' 2024-09-16T10:58:50.415 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi102.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-16T10:58:50.531 DEBUG:teuthology.orchestra.run.smithi102:> true 2024-09-16T10:58:50.608 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi102.front.sepia.ceph.com' 2024-09-16T10:58:50.608 INFO:teuthology.run_tasks:Running task clock... 2024-09-16T10:58:50.619 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-09-16T10:58:50.619 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-16T10:58:50.619 DEBUG:teuthology.orchestra.run.smithi102:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-16T10:58:50.677 INFO:teuthology.orchestra.run.smithi102.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-16T10:58:50.691 INFO:teuthology.orchestra.run.smithi102.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-16T10:58:50.715 INFO:teuthology.orchestra.run.smithi102.stderr:sudo: ntpd: command not found 2024-09-16T10:58:50.725 INFO:teuthology.orchestra.run.smithi102.stdout:506 Cannot talk to daemon 2024-09-16T10:58:50.740 INFO:teuthology.orchestra.run.smithi102.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-16T10:58:50.754 INFO:teuthology.orchestra.run.smithi102.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-16T10:58:50.807 INFO:teuthology.orchestra.run.smithi102.stderr:bash: line 1: ntpq: command not found 2024-09-16T10:58:50.810 INFO:teuthology.orchestra.run.smithi102.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-16T10:58:50.811 INFO:teuthology.orchestra.run.smithi102.stdout:=============================================================================== 2024-09-16T10:58:50.811 INFO:teuthology.orchestra.run.smithi102.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-16T10:58:50.811 INFO:teuthology.orchestra.run.smithi102.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-16T10:58:50.811 INFO:teuthology.orchestra.run.smithi102.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-16T10:58:50.811 INFO:teuthology.orchestra.run.smithi102.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-16T10:58:50.811 INFO:teuthology.run_tasks:Running task install... 2024-09-16T10:58:50.819 DEBUG:teuthology.task.install:project ceph 2024-09-16T10:58:50.820 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'debuginfo': True, 'enable_coprs': ['ceph/el9'], 'flavor': 'default', 'sha1': 'd75f23820806da622f561ee5ee7551015285993f'}} 2024-09-16T10:58:50.820 DEBUG:teuthology.task.install:config {'debuginfo': True, 'enable_coprs': ['ceph/el9'], 'flavor': 'default', 'sha1': 'd75f23820806da622f561ee5ee7551015285993f'} 2024-09-16T10:58:50.820 INFO:teuthology.task.install:Using flavor: default 2024-09-16T10:58:50.826 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse', 'ceph-common-dbg', 'ceph-fuse-dbg', 'ceph-mds-dbg', 'ceph-mgr-dbg', 'ceph-mon-dbg', 'ceph-osd-dbg', 'ceph-test-dbg', 'libcephfs2-dbg', 'librados2-dbg', 'libradosstriper1-dbg', 'librbd1-dbg', 'librgw2-dbg', 'radosgw-dbg', 'rbd-fuse-dbg', 'rbd-mirror-dbg', 'rbd-nbd-dbg'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd', 'ceph-base-debuginfo', 'ceph-common-debuginfo', 'ceph-immutable-object-cache-debuginfo', 'ceph-radosgw-debuginfo', 'ceph-test-debuginfo', 'ceph-base-debuginfo', 'ceph-mgr-debuginfo', 'ceph-mds-debuginfo', 'ceph-mon-debuginfo', 'ceph-osd-debuginfo', 'ceph-fuse-debuginfo', 'librados-devel-debuginfo', 'libcephfs2-debuginfo', 'librados2-debuginfo', 'librbd1-debuginfo', 'python3-cephfs-debuginfo', 'python3-rados-debuginfo', 'python3-rbd-debuginfo', 'python3-rgw-debuginfo', 'rbd-fuse-debuginfo', 'rbd-mirror-debuginfo', 'rbd-nbd-debuginfo']} 2024-09-16T10:58:50.826 INFO:teuthology.task.install:extra packages: [] 2024-09-16T10:58:50.827 DEBUG:teuthology.orchestra.run.smithi102:> sudo dnf -y install 'dnf-command(copr)' 2024-09-16T10:58:51.408 INFO:teuthology.orchestra.run.smithi102.stdout:Last metadata expiration check: 0:00:43 ago on Mon 16 Sep 2024 10:58:08 AM UTC. 2024-09-16T10:58:51.495 INFO:teuthology.orchestra.run.smithi102.stdout:Package dnf-plugins-core-4.3.0-16.el9.noarch is already installed. 2024-09-16T10:58:51.539 INFO:teuthology.orchestra.run.smithi102.stdout:Dependencies resolved. 2024-09-16T10:58:51.540 INFO:teuthology.orchestra.run.smithi102.stdout:Nothing to do. 2024-09-16T10:58:51.540 INFO:teuthology.orchestra.run.smithi102.stdout:Complete! 2024-09-16T10:58:51.614 DEBUG:teuthology.orchestra.run.smithi102:> sudo dnf -y copr enable ceph/el9 2024-09-16T10:58:51.794 INFO:teuthology.orchestra.run.smithi102.stderr:Enabling a Copr repository. Please note that this repository is not part 2024-09-16T10:58:51.795 INFO:teuthology.orchestra.run.smithi102.stderr:of the main distribution, and quality may vary. 2024-09-16T10:58:51.795 INFO:teuthology.orchestra.run.smithi102.stderr: 2024-09-16T10:58:51.795 INFO:teuthology.orchestra.run.smithi102.stderr:The Fedora Project does not exercise any power over the contents of 2024-09-16T10:58:51.795 INFO:teuthology.orchestra.run.smithi102.stderr:this repository beyond the rules outlined in the Copr FAQ at 2024-09-16T10:58:51.795 INFO:teuthology.orchestra.run.smithi102.stderr:, 2024-09-16T10:58:51.795 INFO:teuthology.orchestra.run.smithi102.stderr:and packages are not held to any quality or security level. 2024-09-16T10:58:51.795 INFO:teuthology.orchestra.run.smithi102.stderr: 2024-09-16T10:58:51.795 INFO:teuthology.orchestra.run.smithi102.stderr:Please do not file bug reports about these packages in Fedora 2024-09-16T10:58:51.795 INFO:teuthology.orchestra.run.smithi102.stderr:Bugzilla. In case of problems, contact the owner of this repository. 2024-09-16T10:58:51.846 INFO:teuthology.orchestra.run.smithi102.stdout:Repository successfully enabled. 2024-09-16T10:58:51.881 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': True, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': ['ceph/el9'], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'd75f23820806da622f561ee5ee7551015285993f', 'tag': None, 'wait_for_package': False} 2024-09-16T10:58:51.882 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d75f23820806da622f561ee5ee7551015285993f 2024-09-16T10:58:52.033 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/centos/9/flavors/default/ 2024-09-16T10:58:52.033 INFO:teuthology.task.install.rpm:Package version is 17.2.7-1643.gd75f2382 2024-09-16T10:58:52.143 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-bharath9-testing-2024-09-13-1444-quincy/d75f23820806da622f561ee5ee7551015285993f/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-16T10:58:52.143 DEBUG:teuthology.orchestra.run.smithi102:> set -ex 2024-09-16T10:58:52.143 DEBUG:teuthology.orchestra.run.smithi102:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-16T10:58:52.169 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, ceph-base-debuginfo, ceph-common-debuginfo, ceph-immutable-object-cache-debuginfo, ceph-radosgw-debuginfo, ceph-test-debuginfo, ceph-base-debuginfo, ceph-mgr-debuginfo, ceph-mds-debuginfo, ceph-mon-debuginfo, ceph-osd-debuginfo, ceph-fuse-debuginfo, librados-devel-debuginfo, libcephfs2-debuginfo, librados2-debuginfo, librbd1-debuginfo, python3-cephfs-debuginfo, python3-rados-debuginfo, python3-rbd-debuginfo, python3-rgw-debuginfo, rbd-fuse-debuginfo, rbd-mirror-debuginfo, rbd-nbd-debuginfo on remote rpm x86_64 2024-09-16T10:58:52.169 DEBUG:teuthology.orchestra.run.smithi102:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/d75f23820806da622f561ee5ee7551015285993f/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-16T10:58:52.234 DEBUG:teuthology.orchestra.run.smithi102:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-16T10:58:52.309 DEBUG:teuthology.orchestra.run.smithi102:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-16T10:58:52.374 INFO:teuthology.orchestra.run.smithi102.stdout:check_obsoletes = 1 2024-09-16T10:58:52.375 DEBUG:teuthology.orchestra.run.smithi102:> sudo yum clean all 2024-09-16T10:58:52.606 INFO:teuthology.orchestra.run.smithi102.stdout:45 files removed 2024-09-16T10:58:52.632 DEBUG:teuthology.orchestra.run.smithi102:> sudo yum -y install ceph-radosgw 2024-09-16T10:58:53.196 INFO:teuthology.orchestra.run.smithi102.stdout:ceph packages for x86_64 277 kB/s | 77 kB 00:00 2024-09-16T10:58:53.488 INFO:teuthology.orchestra.run.smithi102.stdout:ceph noarch packages 44 kB/s | 11 kB 00:00 2024-09-16T10:58:53.758 INFO:teuthology.orchestra.run.smithi102.stdout:ceph source packages 7.7 kB/s | 1.8 kB 00:00 2024-09-16T10:58:53.925 INFO:teuthology.orchestra.run.smithi102.stdout:Copr repo for el9 owned by ceph 58 kB/s | 7.6 kB 00:00 2024-09-16T10:58:54.542 INFO:teuthology.orchestra.run.smithi102.stdout:CentOS Stream 9 - BaseOS 14 MB/s | 8.2 MB 00:00 2024-09-16T10:58:56.560 INFO:teuthology.orchestra.run.smithi102.stdout:CentOS Stream 9 - AppStream 23 MB/s | 20 MB 00:00 2024-09-16T10:59:02.236 INFO:teuthology.orchestra.run.smithi102.stdout:CentOS Stream 9 - CRB 4.0 MB/s | 6.5 MB 00:01 2024-09-16T10:59:03.878 INFO:teuthology.orchestra.run.smithi102.stdout:CentOS Stream 9 - Extras packages 98 kB/s | 19 kB 00:00 2024-09-16T10:59:04.732 INFO:teuthology.orchestra.run.smithi102.stdout:Extra Packages for Enterprise Linux 31 MB/s | 23 MB 00:00 2024-09-16T10:59:10.505 INFO:teuthology.orchestra.run.smithi102.stdout:lab-extras 35 kB/s | 1.7 kB 00:00 2024-09-16T10:59:12.989 INFO:teuthology.orchestra.run.smithi102.stdout:Dependencies resolved. 2024-09-16T10:59:13.001 INFO:teuthology.orchestra.run.smithi102.stdout:================================================================================ 2024-09-16T10:59:13.001 INFO:teuthology.orchestra.run.smithi102.stdout: Package Arch Version Repository Size 2024-09-16T10:59:13.001 INFO:teuthology.orchestra.run.smithi102.stdout:================================================================================ 2024-09-16T10:59:13.001 INFO:teuthology.orchestra.run.smithi102.stdout:Installing: 2024-09-16T10:59:13.001 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-radosgw x86_64 2:17.2.7-1643.gd75f2382.el9 ceph 14 M 2024-09-16T10:59:13.001 INFO:teuthology.orchestra.run.smithi102.stdout:Upgrading: 2024-09-16T10:59:13.001 INFO:teuthology.orchestra.run.smithi102.stdout: librados2 x86_64 2:17.2.7-1643.gd75f2382.el9 ceph 3.1 M 2024-09-16T10:59:13.001 INFO:teuthology.orchestra.run.smithi102.stdout: librbd1 x86_64 2:17.2.7-1643.gd75f2382.el9 ceph 2.9 M 2024-09-16T10:59:13.002 INFO:teuthology.orchestra.run.smithi102.stdout:Installing dependencies: 2024-09-16T10:59:13.002 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-base x86_64 2:17.2.7-1643.gd75f2382.el9 ceph 5.0 M 2024-09-16T10:59:13.002 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-common x86_64 2:17.2.7-1643.gd75f2382.el9 ceph 22 M 2024-09-16T10:59:13.002 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-selinux x86_64 2:17.2.7-1643.gd75f2382.el9 ceph 25 k 2024-09-16T10:59:13.002 INFO:teuthology.orchestra.run.smithi102.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-16T10:59:13.002 INFO:teuthology.orchestra.run.smithi102.stdout: libcephfs2 x86_64 2:17.2.7-1643.gd75f2382.el9 ceph 637 k 2024-09-16T10:59:13.002 INFO:teuthology.orchestra.run.smithi102.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-16T10:59:13.002 INFO:teuthology.orchestra.run.smithi102.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-16T10:59:13.002 INFO:teuthology.orchestra.run.smithi102.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-16T10:59:13.002 INFO:teuthology.orchestra.run.smithi102.stdout: libradosstriper1 x86_64 2:17.2.7-1643.gd75f2382.el9 ceph 458 k 2024-09-16T10:59:13.002 INFO:teuthology.orchestra.run.smithi102.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-16T10:59:13.002 INFO:teuthology.orchestra.run.smithi102.stdout: librgw2 x86_64 2:17.2.7-1643.gd75f2382.el9 ceph 4.9 M 2024-09-16T10:59:13.002 INFO:teuthology.orchestra.run.smithi102.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-16T10:59:13.002 INFO:teuthology.orchestra.run.smithi102.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-16T10:59:13.003 INFO:teuthology.orchestra.run.smithi102.stdout: python3-ceph-argparse x86_64 2:17.2.7-1643.gd75f2382.el9 ceph 45 k 2024-09-16T10:59:13.003 INFO:teuthology.orchestra.run.smithi102.stdout: python3-ceph-common x86_64 2:17.2.7-1643.gd75f2382.el9 ceph 120 k 2024-09-16T10:59:13.003 INFO:teuthology.orchestra.run.smithi102.stdout: python3-cephfs x86_64 2:17.2.7-1643.gd75f2382.el9 ceph 156 k 2024-09-16T10:59:13.003 INFO:teuthology.orchestra.run.smithi102.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-16T10:59:13.003 INFO:teuthology.orchestra.run.smithi102.stdout: python3-rados x86_64 2:17.2.7-1643.gd75f2382.el9 ceph 320 k 2024-09-16T10:59:13.003 INFO:teuthology.orchestra.run.smithi102.stdout: python3-rbd x86_64 2:17.2.7-1643.gd75f2382.el9 ceph 295 k 2024-09-16T10:59:13.003 INFO:teuthology.orchestra.run.smithi102.stdout: python3-rgw x86_64 2:17.2.7-1643.gd75f2382.el9 ceph 100 k 2024-09-16T10:59:13.003 INFO:teuthology.orchestra.run.smithi102.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-16T10:59:13.003 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-16T10:59:13.003 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction Summary 2024-09-16T10:59:13.003 INFO:teuthology.orchestra.run.smithi102.stdout:================================================================================ 2024-09-16T10:59:13.003 INFO:teuthology.orchestra.run.smithi102.stdout:Install 22 Packages 2024-09-16T10:59:13.003 INFO:teuthology.orchestra.run.smithi102.stdout:Upgrade 2 Packages 2024-09-16T10:59:13.004 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-16T10:59:13.004 INFO:teuthology.orchestra.run.smithi102.stdout:Total download size: 58 M 2024-09-16T10:59:13.004 INFO:teuthology.orchestra.run.smithi102.stdout:Downloading Packages: 2024-09-16T10:59:14.015 INFO:teuthology.orchestra.run.smithi102.stdout:(1/24): ceph-base-17.2.7-1643.gd75f2382.el9.x86 9.9 MB/s | 5.0 MB 00:00 2024-09-16T10:59:14.065 INFO:teuthology.orchestra.run.smithi102.stdout:(2/24): ceph-selinux-17.2.7-1643.gd75f2382.el9. 503 kB/s | 25 kB 00:00 2024-09-16T10:59:14.165 INFO:teuthology.orchestra.run.smithi102.stdout:(3/24): libcephfs2-17.2.7-1643.gd75f2382.el9.x8 6.2 MB/s | 637 kB 00:00 2024-09-16T10:59:14.274 INFO:teuthology.orchestra.run.smithi102.stdout:(4/24): libradosstriper1-17.2.7-1643.gd75f2382. 4.1 MB/s | 458 kB 00:00 2024-09-16T10:59:14.449 INFO:teuthology.orchestra.run.smithi102.stdout:(5/24): ceph-radosgw-17.2.7-1643.gd75f2382.el9. 15 MB/s | 14 MB 00:00 2024-09-16T10:59:14.491 INFO:teuthology.orchestra.run.smithi102.stdout:(6/24): python3-ceph-argparse-17.2.7-1643.gd75f 1.1 MB/s | 45 kB 00:00 2024-09-16T10:59:14.566 INFO:teuthology.orchestra.run.smithi102.stdout:(7/24): librgw2-17.2.7-1643.gd75f2382.el9.x86_6 17 MB/s | 4.9 MB 00:00 2024-09-16T10:59:14.583 INFO:teuthology.orchestra.run.smithi102.stdout:(8/24): python3-ceph-common-17.2.7-1643.gd75f23 1.3 MB/s | 120 kB 00:00 2024-09-16T10:59:14.637 INFO:teuthology.orchestra.run.smithi102.stdout:(9/24): python3-cephfs-17.2.7-1643.gd75f2382.el 2.2 MB/s | 156 kB 00:00 2024-09-16T10:59:14.667 INFO:teuthology.orchestra.run.smithi102.stdout:(10/24): python3-rados-17.2.7-1643.gd75f2382.el 3.7 MB/s | 320 kB 00:00 2024-09-16T10:59:14.692 INFO:teuthology.orchestra.run.smithi102.stdout:(11/24): python3-rbd-17.2.7-1643.gd75f2382.el9. 5.3 MB/s | 295 kB 00:00 2024-09-16T10:59:14.718 INFO:teuthology.orchestra.run.smithi102.stdout:(12/24): python3-rgw-17.2.7-1643.gd75f2382.el9. 1.9 MB/s | 100 kB 00:00 2024-09-16T10:59:14.927 INFO:teuthology.orchestra.run.smithi102.stdout:(13/24): ceph-common-17.2.7-1643.gd75f2382.el9. 16 MB/s | 22 MB 00:01 2024-09-16T10:59:15.002 INFO:teuthology.orchestra.run.smithi102.stdout:(14/24): librabbitmq-0.11.0-7.el9.x86_64.rpm 159 kB/s | 45 kB 00:00 2024-09-16T10:59:15.036 INFO:teuthology.orchestra.run.smithi102.stdout:(15/24): libpmemobj-1.12.1-1.el9.x86_64.rpm 467 kB/s | 160 kB 00:00 2024-09-16T10:59:15.078 INFO:teuthology.orchestra.run.smithi102.stdout:(16/24): python3-prettytable-0.7.2-27.el9.noarc 1.0 MB/s | 42 kB 00:00 2024-09-16T10:59:15.103 INFO:teuthology.orchestra.run.smithi102.stdout:(17/24): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.8 MB/s | 292 kB 00:00 2024-09-16T10:59:15.211 INFO:teuthology.orchestra.run.smithi102.stdout:(18/24): librdkafka-1.6.1-102.el9.x86_64.rpm 2.3 MB/s | 662 kB 00:00 2024-09-16T10:59:15.245 INFO:teuthology.orchestra.run.smithi102.stdout:(19/24): liboath-2.6.7-2.el9.x86_64.rpm 342 kB/s | 49 kB 00:00 2024-09-16T10:59:15.270 INFO:teuthology.orchestra.run.smithi102.stdout:(20/24): gperftools-libs-2.9.1-3.el9.x86_64.rpm 1.6 MB/s | 308 kB 00:00 2024-09-16T10:59:15.295 INFO:teuthology.orchestra.run.smithi102.stdout:(21/24): libunwind-1.6.2-1.el9.x86_64.rpm 822 kB/s | 67 kB 00:00 2024-09-16T10:59:15.337 INFO:teuthology.orchestra.run.smithi102.stdout:(22/24): thrift-0.15.0-2.el9.x86_64.rpm 18 MB/s | 1.6 MB 00:00 2024-09-16T10:59:15.454 INFO:teuthology.orchestra.run.smithi102.stdout:(23/24): librados2-17.2.7-1643.gd75f2382.el9.x8 17 MB/s | 3.1 MB 00:00 2024-09-16T10:59:15.504 INFO:teuthology.orchestra.run.smithi102.stdout:(24/24): librbd1-17.2.7-1643.gd75f2382.el9.x86_ 14 MB/s | 2.9 MB 00:00 2024-09-16T10:59:15.509 INFO:teuthology.orchestra.run.smithi102.stdout:-------------------------------------------------------------------------------- 2024-09-16T10:59:15.509 INFO:teuthology.orchestra.run.smithi102.stdout:Total 23 MB/s | 58 MB 00:02 2024-09-16T10:59:15.571 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction check 2024-09-16T10:59:15.605 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction check succeeded. 2024-09-16T10:59:15.605 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction test 2024-09-16T10:59:16.271 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction test succeeded. 2024-09-16T10:59:16.272 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction 2024-09-16T10:59:17.539 INFO:teuthology.orchestra.run.smithi102.stdout: Preparing : 1/1 2024-09-16T10:59:17.919 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/26 2024-09-16T10:59:17.990 INFO:teuthology.orchestra.run.smithi102.stdout: Upgrading : librados2-2:17.2.7-1643.gd75f2382.el9.x86_64 2/26 2024-09-16T10:59:18.083 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: librados2-2:17.2.7-1643.gd75f2382.el9.x86_64 2/26 2024-09-16T10:59:18.161 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-rados-2:17.2.7-1643.gd75f2382.el9.x86_64 3/26 2024-09-16T10:59:18.191 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : libcephfs2-2:17.2.7-1643.gd75f2382.el9.x86_64 4/26 2024-09-16T10:59:18.279 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: libcephfs2-2:17.2.7-1643.gd75f2382.el9.x86_64 4/26 2024-09-16T10:59:18.330 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : thrift-0.15.0-2.el9.x86_64 5/26 2024-09-16T10:59:18.363 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 6/26 2024-09-16T10:59:18.802 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 7/26 2024-09-16T10:59:18.827 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : librgw2-2:17.2.7-1643.gd75f2382.el9.x86_64 8/26 2024-09-16T10:59:18.900 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: librgw2-2:17.2.7-1643.gd75f2382.el9.x86_64 8/26 2024-09-16T10:59:18.927 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : liboath-2.6.7-2.el9.x86_64 9/26 2024-09-16T10:59:18.962 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-ceph-argparse-2:17.2.7-1643.gd75f2382.el9. 10/26 2024-09-16T10:59:18.999 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-cephfs-2:17.2.7-1643.gd75f2382.el9.x86_64 11/26 2024-09-16T10:59:19.062 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-rgw-2:17.2.7-1643.gd75f2382.el9.x86_64 12/26 2024-09-16T10:59:19.088 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : libradosstriper1-2:17.2.7-1643.gd75f2382.el9.x86_6 13/26 2024-09-16T10:59:19.145 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: libradosstriper1-2:17.2.7-1643.gd75f2382.el9.x86_6 13/26 2024-09-16T10:59:19.194 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 14/26 2024-09-16T10:59:19.255 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 15/26 2024-09-16T10:59:19.288 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 16/26 2024-09-16T10:59:19.581 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 17/26 2024-09-16T10:59:19.634 INFO:teuthology.orchestra.run.smithi102.stdout: Upgrading : librbd1-2:17.2.7-1643.gd75f2382.el9.x86_64 18/26 2024-09-16T10:59:19.725 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: librbd1-2:17.2.7-1643.gd75f2382.el9.x86_64 18/26 2024-09-16T10:59:19.797 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-rbd-2:17.2.7-1643.gd75f2382.el9.x86_64 19/26 2024-09-16T10:59:19.827 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-ceph-common-2:17.2.7-1643.gd75f2382.el9.x8 20/26 2024-09-16T10:59:22.338 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: ceph-common-2:17.2.7-1643.gd75f2382.el9.x86_64 21/26 2024-09-16T10:59:22.374 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : ceph-common-2:17.2.7-1643.gd75f2382.el9.x86_64 21/26 2024-09-16T10:59:22.910 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: ceph-common-2:17.2.7-1643.gd75f2382.el9.x86_64 21/26 2024-09-16T10:59:22.952 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : ceph-base-2:17.2.7-1643.gd75f2382.el9.x86_64 22/26 2024-09-16T10:59:23.037 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: ceph-base-2:17.2.7-1643.gd75f2382.el9.x86_64 22/26 2024-09-16T10:59:23.037 INFO:teuthology.orchestra.run.smithi102.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-16T10:59:23.037 INFO:teuthology.orchestra.run.smithi102.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-16T10:59:23.037 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-16T10:59:23.078 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : ceph-selinux-2:17.2.7-1643.gd75f2382.el9.x86_64 23/26 2024-09-16T10:59:33.040 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: ceph-selinux-2:17.2.7-1643.gd75f2382.el9.x86_64 23/26 2024-09-16T10:59:33.041 INFO:teuthology.orchestra.run.smithi102.stdout:skipping the directory /sys 2024-09-16T10:59:33.041 INFO:teuthology.orchestra.run.smithi102.stdout:skipping the directory /proc 2024-09-16T10:59:33.041 INFO:teuthology.orchestra.run.smithi102.stdout:skipping the directory /mnt 2024-09-16T10:59:33.041 INFO:teuthology.orchestra.run.smithi102.stdout:skipping the directory /var/tmp 2024-09-16T10:59:33.041 INFO:teuthology.orchestra.run.smithi102.stdout:skipping the directory /home 2024-09-16T10:59:33.041 INFO:teuthology.orchestra.run.smithi102.stdout:skipping the directory /root 2024-09-16T10:59:33.041 INFO:teuthology.orchestra.run.smithi102.stdout:skipping the directory /tmp 2024-09-16T10:59:33.041 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-16T10:59:34.362 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : ceph-radosgw-2:17.2.7-1643.gd75f2382.el9.x86_64 24/26 2024-09-16T10:59:34.438 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: ceph-radosgw-2:17.2.7-1643.gd75f2382.el9.x86_64 24/26 2024-09-16T10:59:34.438 INFO:teuthology.orchestra.run.smithi102.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T10:59:34.438 INFO:teuthology.orchestra.run.smithi102.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-16T10:59:34.438 INFO:teuthology.orchestra.run.smithi102.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-16T10:59:34.439 INFO:teuthology.orchestra.run.smithi102.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-16T10:59:34.439 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-16T10:59:34.440 INFO:teuthology.orchestra.run.smithi102.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 25/26 2024-09-16T10:59:34.523 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 25/26 2024-09-16T10:59:34.523 INFO:teuthology.orchestra.run.smithi102.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 26/26 2024-09-16T10:59:35.817 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 26/26 2024-09-16T10:59:35.818 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : ceph-base-2:17.2.7-1643.gd75f2382.el9.x86_64 1/26 2024-09-16T10:59:35.818 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : ceph-common-2:17.2.7-1643.gd75f2382.el9.x86_64 2/26 2024-09-16T10:59:35.818 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : ceph-radosgw-2:17.2.7-1643.gd75f2382.el9.x86_64 3/26 2024-09-16T10:59:35.818 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : ceph-selinux-2:17.2.7-1643.gd75f2382.el9.x86_64 4/26 2024-09-16T10:59:35.818 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : libcephfs2-2:17.2.7-1643.gd75f2382.el9.x86_64 5/26 2024-09-16T10:59:35.818 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : libradosstriper1-2:17.2.7-1643.gd75f2382.el9.x86_6 6/26 2024-09-16T10:59:35.818 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : librgw2-2:17.2.7-1643.gd75f2382.el9.x86_64 7/26 2024-09-16T10:59:35.818 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-ceph-argparse-2:17.2.7-1643.gd75f2382.el9. 8/26 2024-09-16T10:59:35.818 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-ceph-common-2:17.2.7-1643.gd75f2382.el9.x8 9/26 2024-09-16T10:59:35.818 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-cephfs-2:17.2.7-1643.gd75f2382.el9.x86_64 10/26 2024-09-16T10:59:35.818 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-rados-2:17.2.7-1643.gd75f2382.el9.x86_64 11/26 2024-09-16T10:59:35.818 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-rbd-2:17.2.7-1643.gd75f2382.el9.x86_64 12/26 2024-09-16T10:59:35.818 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-rgw-2:17.2.7-1643.gd75f2382.el9.x86_64 13/26 2024-09-16T10:59:35.819 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 14/26 2024-09-16T10:59:35.819 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 15/26 2024-09-16T10:59:35.819 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 16/26 2024-09-16T10:59:35.819 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 17/26 2024-09-16T10:59:35.819 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 18/26 2024-09-16T10:59:35.819 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 19/26 2024-09-16T10:59:35.819 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 20/26 2024-09-16T10:59:35.819 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 21/26 2024-09-16T10:59:35.819 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 22/26 2024-09-16T10:59:35.819 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : librados2-2:17.2.7-1643.gd75f2382.el9.x86_64 23/26 2024-09-16T10:59:35.819 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 24/26 2024-09-16T10:59:35.819 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : librbd1-2:17.2.7-1643.gd75f2382.el9.x86_64 25/26 2024-09-16T10:59:36.893 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 26/26 2024-09-16T10:59:36.893 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-16T10:59:36.893 INFO:teuthology.orchestra.run.smithi102.stdout:Upgraded: 2024-09-16T10:59:36.893 INFO:teuthology.orchestra.run.smithi102.stdout: librados2-2:17.2.7-1643.gd75f2382.el9.x86_64 2024-09-16T10:59:36.893 INFO:teuthology.orchestra.run.smithi102.stdout: librbd1-2:17.2.7-1643.gd75f2382.el9.x86_64 2024-09-16T10:59:36.893 INFO:teuthology.orchestra.run.smithi102.stdout:Installed: 2024-09-16T10:59:36.893 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-base-2:17.2.7-1643.gd75f2382.el9.x86_64 2024-09-16T10:59:36.893 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-common-2:17.2.7-1643.gd75f2382.el9.x86_64 2024-09-16T10:59:36.893 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-radosgw-2:17.2.7-1643.gd75f2382.el9.x86_64 2024-09-16T10:59:36.894 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-selinux-2:17.2.7-1643.gd75f2382.el9.x86_64 2024-09-16T10:59:36.894 INFO:teuthology.orchestra.run.smithi102.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-16T10:59:36.894 INFO:teuthology.orchestra.run.smithi102.stdout: libcephfs2-2:17.2.7-1643.gd75f2382.el9.x86_64 2024-09-16T10:59:36.894 INFO:teuthology.orchestra.run.smithi102.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-16T10:59:36.894 INFO:teuthology.orchestra.run.smithi102.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-16T10:59:36.894 INFO:teuthology.orchestra.run.smithi102.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-16T10:59:36.894 INFO:teuthology.orchestra.run.smithi102.stdout: libradosstriper1-2:17.2.7-1643.gd75f2382.el9.x86_64 2024-09-16T10:59:36.894 INFO:teuthology.orchestra.run.smithi102.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-16T10:59:36.894 INFO:teuthology.orchestra.run.smithi102.stdout: librgw2-2:17.2.7-1643.gd75f2382.el9.x86_64 2024-09-16T10:59:36.894 INFO:teuthology.orchestra.run.smithi102.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-16T10:59:36.894 INFO:teuthology.orchestra.run.smithi102.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-16T10:59:36.894 INFO:teuthology.orchestra.run.smithi102.stdout: python3-ceph-argparse-2:17.2.7-1643.gd75f2382.el9.x86_64 2024-09-16T10:59:36.895 INFO:teuthology.orchestra.run.smithi102.stdout: python3-ceph-common-2:17.2.7-1643.gd75f2382.el9.x86_64 2024-09-16T10:59:36.895 INFO:teuthology.orchestra.run.smithi102.stdout: python3-cephfs-2:17.2.7-1643.gd75f2382.el9.x86_64 2024-09-16T10:59:36.895 INFO:teuthology.orchestra.run.smithi102.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-16T10:59:36.895 INFO:teuthology.orchestra.run.smithi102.stdout: python3-rados-2:17.2.7-1643.gd75f2382.el9.x86_64 2024-09-16T10:59:36.895 INFO:teuthology.orchestra.run.smithi102.stdout: python3-rbd-2:17.2.7-1643.gd75f2382.el9.x86_64 2024-09-16T10:59:36.895 INFO:teuthology.orchestra.run.smithi102.stdout: python3-rgw-2:17.2.7-1643.gd75f2382.el9.x86_64 2024-09-16T10:59:36.895 INFO:teuthology.orchestra.run.smithi102.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-16T10:59:36.895 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-16T10:59:36.895 INFO:teuthology.orchestra.run.smithi102.stdout:Complete! 2024-09-16T10:59:37.203 DEBUG:teuthology.orchestra.run.smithi102:> sudo yum -y install ceph-test 2024-09-16T10:59:37.766 INFO:teuthology.orchestra.run.smithi102.stdout:Last metadata expiration check: 0:00:27 ago on Mon 16 Sep 2024 10:59:10 AM UTC. 2024-09-16T10:59:37.901 INFO:teuthology.orchestra.run.smithi102.stdout:Dependencies resolved. 2024-09-16T10:59:37.902 INFO:teuthology.orchestra.run.smithi102.stdout:================================================================================ 2024-09-16T10:59:37.902 INFO:teuthology.orchestra.run.smithi102.stdout: Package Arch Version Repository Size 2024-09-16T10:59:37.902 INFO:teuthology.orchestra.run.smithi102.stdout:================================================================================ 2024-09-16T10:59:37.902 INFO:teuthology.orchestra.run.smithi102.stdout:Installing: 2024-09-16T10:59:37.902 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-test x86_64 2:17.2.7-1643.gd75f2382.el9 ceph 48 M 2024-09-16T10:59:37.902 INFO:teuthology.orchestra.run.smithi102.stdout:Installing dependencies: 2024-09-16T10:59:37.902 INFO:teuthology.orchestra.run.smithi102.stdout: libcephsqlite x86_64 2:17.2.7-1643.gd75f2382.el9 ceph 163 k 2024-09-16T10:59:37.902 INFO:teuthology.orchestra.run.smithi102.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-16T10:59:37.902 INFO:teuthology.orchestra.run.smithi102.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-16T10:59:37.902 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-16T10:59:37.902 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction Summary 2024-09-16T10:59:37.903 INFO:teuthology.orchestra.run.smithi102.stdout:================================================================================ 2024-09-16T10:59:37.903 INFO:teuthology.orchestra.run.smithi102.stdout:Install 4 Packages 2024-09-16T10:59:37.903 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-16T10:59:37.903 INFO:teuthology.orchestra.run.smithi102.stdout:Total download size: 48 M 2024-09-16T10:59:37.904 INFO:teuthology.orchestra.run.smithi102.stdout:Installed size: 193 M 2024-09-16T10:59:37.904 INFO:teuthology.orchestra.run.smithi102.stdout:Downloading Packages: 2024-09-16T10:59:38.139 INFO:teuthology.orchestra.run.smithi102.stdout:(1/4): libcephsqlite-17.2.7-1643.gd75f2382.el9. 757 kB/s | 163 kB 00:00 2024-09-16T10:59:38.164 INFO:teuthology.orchestra.run.smithi102.stdout:(2/4): socat-1.7.4.1-6.el9.x86_64.rpm 1.2 MB/s | 304 kB 00:00 2024-09-16T10:59:38.281 INFO:teuthology.orchestra.run.smithi102.stdout:(3/4): xmlstarlet-1.6.1-20.el9.x86_64.rpm 450 kB/s | 64 kB 00:00 2024-09-16T10:59:39.440 INFO:teuthology.orchestra.run.smithi102.stdout:(4/4): ceph-test-17.2.7-1643.gd75f2382.el9.x86_ 32 MB/s | 48 MB 00:01 2024-09-16T10:59:39.442 INFO:teuthology.orchestra.run.smithi102.stdout:-------------------------------------------------------------------------------- 2024-09-16T10:59:39.442 INFO:teuthology.orchestra.run.smithi102.stdout:Total 31 MB/s | 48 MB 00:01 2024-09-16T10:59:39.468 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction check 2024-09-16T10:59:39.487 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction check succeeded. 2024-09-16T10:59:39.487 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction test 2024-09-16T10:59:39.855 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction test succeeded. 2024-09-16T10:59:39.856 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction 2024-09-16T10:59:40.384 INFO:teuthology.orchestra.run.smithi102.stdout: Preparing : 1/1 2024-09-16T10:59:40.459 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 1/4 2024-09-16T10:59:40.498 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 2/4 2024-09-16T10:59:40.525 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : libcephsqlite-2:17.2.7-1643.gd75f2382.el9.x86_64 3/4 2024-09-16T10:59:44.964 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: libcephsqlite-2:17.2.7-1643.gd75f2382.el9.x86_64 3/4 2024-09-16T10:59:45.048 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : ceph-test-2:17.2.7-1643.gd75f2382.el9.x86_64 4/4 2024-09-16T10:59:47.332 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: ceph-test-2:17.2.7-1643.gd75f2382.el9.x86_64 4/4 2024-09-16T10:59:47.332 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : ceph-test-2:17.2.7-1643.gd75f2382.el9.x86_64 1/4 2024-09-16T10:59:47.332 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : libcephsqlite-2:17.2.7-1643.gd75f2382.el9.x86_64 2/4 2024-09-16T10:59:47.332 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 3/4 2024-09-16T10:59:47.681 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 4/4 2024-09-16T10:59:47.681 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-16T10:59:47.681 INFO:teuthology.orchestra.run.smithi102.stdout:Installed: 2024-09-16T10:59:47.681 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-test-2:17.2.7-1643.gd75f2382.el9.x86_64 2024-09-16T10:59:47.681 INFO:teuthology.orchestra.run.smithi102.stdout: libcephsqlite-2:17.2.7-1643.gd75f2382.el9.x86_64 2024-09-16T10:59:47.681 INFO:teuthology.orchestra.run.smithi102.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-16T10:59:47.681 INFO:teuthology.orchestra.run.smithi102.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-16T10:59:47.681 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-16T10:59:47.682 INFO:teuthology.orchestra.run.smithi102.stdout:Complete! 2024-09-16T10:59:47.986 DEBUG:teuthology.orchestra.run.smithi102:> sudo yum -y install ceph 2024-09-16T10:59:48.560 INFO:teuthology.orchestra.run.smithi102.stdout:Last metadata expiration check: 0:00:38 ago on Mon 16 Sep 2024 10:59:10 AM UTC. 2024-09-16T10:59:48.703 INFO:teuthology.orchestra.run.smithi102.stdout:Dependencies resolved. 2024-09-16T10:59:48.705 INFO:teuthology.orchestra.run.smithi102.stdout:================================================================================ 2024-09-16T10:59:48.705 INFO:teuthology.orchestra.run.smithi102.stdout: Package Arch Version Repository Size 2024-09-16T10:59:48.705 INFO:teuthology.orchestra.run.smithi102.stdout:================================================================================ 2024-09-16T10:59:48.706 INFO:teuthology.orchestra.run.smithi102.stdout:Installing: 2024-09-16T10:59:48.706 INFO:teuthology.orchestra.run.smithi102.stdout: ceph x86_64 2:17.2.7-1643.gd75f2382.el9 ceph 6.5 k 2024-09-16T10:59:48.706 INFO:teuthology.orchestra.run.smithi102.stdout:Installing dependencies: 2024-09-16T10:59:48.706 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-mds x86_64 2:17.2.7-1643.gd75f2382.el9 ceph 2.1 M 2024-09-16T10:59:48.706 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-mgr x86_64 2:17.2.7-1643.gd75f2382.el9 ceph 1.4 M 2024-09-16T10:59:48.706 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-mgr-modules-core noarch 2:17.2.7-1643.gd75f2382.el9 ceph-noarch 240 k 2024-09-16T10:59:48.706 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-mon x86_64 2:17.2.7-1643.gd75f2382.el9 ceph 3.9 M 2024-09-16T10:59:48.706 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-osd x86_64 2:17.2.7-1643.gd75f2382.el9 ceph 15 M 2024-09-16T10:59:48.706 INFO:teuthology.orchestra.run.smithi102.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-16T10:59:48.706 INFO:teuthology.orchestra.run.smithi102.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-16T10:59:48.706 INFO:teuthology.orchestra.run.smithi102.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-16T10:59:48.706 INFO:teuthology.orchestra.run.smithi102.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-16T10:59:48.706 INFO:teuthology.orchestra.run.smithi102.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-16T10:59:48.707 INFO:teuthology.orchestra.run.smithi102.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-16T10:59:48.707 INFO:teuthology.orchestra.run.smithi102.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-16T10:59:48.707 INFO:teuthology.orchestra.run.smithi102.stdout: python3-jaraco-collections 2024-09-16T10:59:48.707 INFO:teuthology.orchestra.run.smithi102.stdout: noarch 3.0.0-8.el9 epel 23 k 2024-09-16T10:59:48.707 INFO:teuthology.orchestra.run.smithi102.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-16T10:59:48.707 INFO:teuthology.orchestra.run.smithi102.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-16T10:59:48.707 INFO:teuthology.orchestra.run.smithi102.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-16T10:59:48.707 INFO:teuthology.orchestra.run.smithi102.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-16T10:59:48.707 INFO:teuthology.orchestra.run.smithi102.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-16T10:59:48.707 INFO:teuthology.orchestra.run.smithi102.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-16T10:59:48.707 INFO:teuthology.orchestra.run.smithi102.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-16T10:59:48.707 INFO:teuthology.orchestra.run.smithi102.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-16T10:59:48.707 INFO:teuthology.orchestra.run.smithi102.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-16T10:59:48.707 INFO:teuthology.orchestra.run.smithi102.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-16T10:59:48.708 INFO:teuthology.orchestra.run.smithi102.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-16T10:59:48.708 INFO:teuthology.orchestra.run.smithi102.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-16T10:59:48.708 INFO:teuthology.orchestra.run.smithi102.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-16T10:59:48.708 INFO:teuthology.orchestra.run.smithi102.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-16T10:59:48.708 INFO:teuthology.orchestra.run.smithi102.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-16T10:59:48.708 INFO:teuthology.orchestra.run.smithi102.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-16T10:59:48.708 INFO:teuthology.orchestra.run.smithi102.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-16T10:59:48.708 INFO:teuthology.orchestra.run.smithi102.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-16T10:59:48.708 INFO:teuthology.orchestra.run.smithi102.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-16T10:59:48.708 INFO:teuthology.orchestra.run.smithi102.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-16T10:59:48.708 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-16T10:59:48.708 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction Summary 2024-09-16T10:59:48.709 INFO:teuthology.orchestra.run.smithi102.stdout:================================================================================ 2024-09-16T10:59:48.709 INFO:teuthology.orchestra.run.smithi102.stdout:Install 34 Packages 2024-09-16T10:59:48.709 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-16T10:59:48.709 INFO:teuthology.orchestra.run.smithi102.stdout:Total download size: 27 M 2024-09-16T10:59:48.709 INFO:teuthology.orchestra.run.smithi102.stdout:Installed size: 96 M 2024-09-16T10:59:48.710 INFO:teuthology.orchestra.run.smithi102.stdout:Downloading Packages: 2024-09-16T10:59:49.507 INFO:teuthology.orchestra.run.smithi102.stdout:(1/34): ceph-17.2.7-1643.gd75f2382.el9.x86_64.r 53 kB/s | 6.5 kB 00:00 2024-09-16T10:59:49.699 INFO:teuthology.orchestra.run.smithi102.stdout:(2/34): ceph-mgr-17.2.7-1643.gd75f2382.el9.x86_ 4.5 MB/s | 1.4 MB 00:00 2024-09-16T10:59:49.749 INFO:teuthology.orchestra.run.smithi102.stdout:(3/34): ceph-mds-17.2.7-1643.gd75f2382.el9.x86_ 5.7 MB/s | 2.1 MB 00:00 2024-09-16T10:59:49.824 INFO:teuthology.orchestra.run.smithi102.stdout:(4/34): ceph-mon-17.2.7-1643.gd75f2382.el9.x86_ 12 MB/s | 3.9 MB 00:00 2024-09-16T10:59:49.858 INFO:teuthology.orchestra.run.smithi102.stdout:(5/34): ceph-mgr-modules-core-17.2.7-1643.gd75f 2.2 MB/s | 240 kB 00:00 2024-09-16T10:59:50.017 INFO:teuthology.orchestra.run.smithi102.stdout:(6/34): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.3 MB/s | 253 kB 00:00 2024-09-16T10:59:50.067 INFO:teuthology.orchestra.run.smithi102.stdout:(7/34): python3-ply-3.11-14.el9.noarch.rpm 2.1 MB/s | 106 kB 00:00 2024-09-16T10:59:50.234 INFO:teuthology.orchestra.run.smithi102.stdout:(8/34): ceph-osd-17.2.7-1643.gd75f2382.el9.x86_ 29 MB/s | 15 MB 00:00 2024-09-16T10:59:50.259 INFO:teuthology.orchestra.run.smithi102.stdout:(9/34): python3-pycparser-2.20-6.el9.noarch.rpm 704 kB/s | 135 kB 00:00 2024-09-16T10:59:50.297 INFO:teuthology.orchestra.run.smithi102.stdout:(10/34): python3-cryptography-36.0.1-4.el9.x86_ 2.8 MB/s | 1.2 MB 00:00 2024-09-16T10:59:50.318 INFO:teuthology.orchestra.run.smithi102.stdout:(11/34): python3-urllib3-1.26.5-6.el9.noarch.rp 3.6 MB/s | 215 kB 00:00 2024-09-16T10:59:50.377 INFO:teuthology.orchestra.run.smithi102.stdout:(12/34): python3-requests-2.25.1-8.el9.noarch.r 880 kB/s | 125 kB 00:00 2024-09-16T10:59:50.393 INFO:teuthology.orchestra.run.smithi102.stdout:(13/34): python3-markupsafe-1.1.1-12.el9.x86_64 462 kB/s | 35 kB 00:00 2024-09-16T10:59:50.419 INFO:teuthology.orchestra.run.smithi102.stdout:(14/34): python3-mako-1.1.4-6.el9.noarch.rpm 1.3 MB/s | 172 kB 00:00 2024-09-16T10:59:50.444 INFO:teuthology.orchestra.run.smithi102.stdout:(15/34): python3-pytz-2021.1-5.el9.noarch.rpm 1.0 MB/s | 51 kB 00:00 2024-09-16T10:59:50.461 INFO:teuthology.orchestra.run.smithi102.stdout:(16/34): python3-toml-0.10.2-6.el9.noarch.rpm 996 kB/s | 42 kB 00:00 2024-09-16T10:59:50.503 INFO:teuthology.orchestra.run.smithi102.stdout:(17/34): python3-packaging-20.9-5.el9.noarch.rp 612 kB/s | 77 kB 00:00 2024-09-16T10:59:50.520 INFO:teuthology.orchestra.run.smithi102.stdout:(18/34): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 577 kB/s | 43 kB 00:00 2024-09-16T10:59:50.554 INFO:teuthology.orchestra.run.smithi102.stdout:(19/34): python3-jaraco-8.2.1-3.el9.noarch.rpm 407 kB/s | 11 kB 00:00 2024-09-16T10:59:50.579 INFO:teuthology.orchestra.run.smithi102.stdout:(20/34): python3-cheroot-8.6.0-4.el9.noarch.rpm 1.4 MB/s | 172 kB 00:00 2024-09-16T10:59:50.604 INFO:teuthology.orchestra.run.smithi102.stdout:(21/34): python3-jaraco-classes-3.2.1-5.el9.noa 364 kB/s | 18 kB 00:00 2024-09-16T10:59:50.630 INFO:teuthology.orchestra.run.smithi102.stdout:(22/34): python3-jaraco-collections-3.0.0-8.el9 463 kB/s | 23 kB 00:00 2024-09-16T10:59:50.655 INFO:teuthology.orchestra.run.smithi102.stdout:(23/34): python3-jaraco-functools-3.5.0-2.el9.n 388 kB/s | 19 kB 00:00 2024-09-16T10:59:50.680 INFO:teuthology.orchestra.run.smithi102.stdout:(24/34): python3-jaraco-text-3.2.0-6.el9.noarch 393 kB/s | 20 kB 00:00 2024-09-16T10:59:50.705 INFO:teuthology.orchestra.run.smithi102.stdout:(25/34): python3-logutils-0.3.5-21.el9.noarch.r 923 kB/s | 46 kB 00:00 2024-09-16T10:59:50.731 INFO:teuthology.orchestra.run.smithi102.stdout:(26/34): python3-more-itertools-8.12.0-2.el9.no 1.5 MB/s | 79 kB 00:00 2024-09-16T10:59:50.756 INFO:teuthology.orchestra.run.smithi102.stdout:(27/34): python3-cherrypy-18.6.1-2.el9.noarch.r 1.4 MB/s | 358 kB 00:00 2024-09-16T10:59:50.781 INFO:teuthology.orchestra.run.smithi102.stdout:(28/34): python3-portend-3.1.0-2.el9.noarch.rpm 328 kB/s | 16 kB 00:00 2024-09-16T10:59:50.807 INFO:teuthology.orchestra.run.smithi102.stdout:(29/34): python3-pyOpenSSL-21.0.0-1.el9.noarch. 1.8 MB/s | 90 kB 00:00 2024-09-16T10:59:50.832 INFO:teuthology.orchestra.run.smithi102.stdout:(30/34): python3-pecan-1.4.2-3.el9.noarch.rpm 2.1 MB/s | 272 kB 00:00 2024-09-16T10:59:50.857 INFO:teuthology.orchestra.run.smithi102.stdout:(31/34): python3-tempora-5.0.0-2.el9.noarch.rpm 474 kB/s | 36 kB 00:00 2024-09-16T10:59:50.882 INFO:teuthology.orchestra.run.smithi102.stdout:(32/34): python3-zc-lockfile-2.0-10.el9.noarch. 798 kB/s | 20 kB 00:00 2024-09-16T10:59:50.910 INFO:teuthology.orchestra.run.smithi102.stdout:(33/34): python3-webob-1.8.8-2.el9.noarch.rpm 2.2 MB/s | 230 kB 00:00 2024-09-16T10:59:50.933 INFO:teuthology.orchestra.run.smithi102.stdout:(34/34): python3-werkzeug-2.0.3-3.el9.1.noarch. 4.1 MB/s | 427 kB 00:00 2024-09-16T10:59:50.936 INFO:teuthology.orchestra.run.smithi102.stdout:-------------------------------------------------------------------------------- 2024-09-16T10:59:50.936 INFO:teuthology.orchestra.run.smithi102.stdout:Total 12 MB/s | 27 MB 00:02 2024-09-16T10:59:51.068 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction check 2024-09-16T10:59:51.105 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction check succeeded. 2024-09-16T10:59:51.105 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction test 2024-09-16T10:59:51.513 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction test succeeded. 2024-09-16T10:59:51.513 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction 2024-09-16T10:59:52.795 INFO:teuthology.orchestra.run.smithi102.stdout: Preparing : 1/1 2024-09-16T10:59:52.861 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/34 2024-09-16T10:59:52.893 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/34 2024-09-16T10:59:52.926 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 3/34 2024-09-16T10:59:53.019 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 4/34 2024-09-16T10:59:53.097 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 5/34 2024-09-16T10:59:53.137 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 6/34 2024-09-16T10:59:53.181 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 7/34 2024-09-16T10:59:53.210 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 8/34 2024-09-16T10:59:53.245 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 9/34 2024-09-16T10:59:53.278 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 10/34 2024-09-16T10:59:53.319 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 11/34 2024-09-16T10:59:53.354 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 12/34 2024-09-16T10:59:53.388 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 13/34 2024-09-16T10:59:53.436 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 14/34 2024-09-16T10:59:53.472 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-packaging-20.9-5.el9.noarch 15/34 2024-09-16T10:59:53.546 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 16/34 2024-09-16T10:59:53.685 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 17/34 2024-09-16T10:59:53.764 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 18/34 2024-09-16T10:59:53.817 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 19/34 2024-09-16T10:59:53.855 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 20/34 2024-09-16T10:59:53.904 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-ply-3.11-14.el9.noarch 21/34 2024-09-16T10:59:53.959 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 22/34 2024-09-16T10:59:54.145 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 23/34 2024-09-16T10:59:54.201 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 24/34 2024-09-16T10:59:54.279 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 25/34 2024-09-16T10:59:54.367 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 26/34 2024-09-16T10:59:54.400 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 27/34 2024-09-16T10:59:54.524 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 28/34 2024-09-16T10:59:54.688 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : ceph-mgr-modules-core-2:17.2.7-1643.gd75f2382.el9. 29/34 2024-09-16T10:59:54.719 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : ceph-mgr-2:17.2.7-1643.gd75f2382.el9.x86_64 30/34 2024-09-16T10:59:54.745 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: ceph-mgr-2:17.2.7-1643.gd75f2382.el9.x86_64 30/34 2024-09-16T10:59:54.745 INFO:teuthology.orchestra.run.smithi102.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T10:59:54.745 INFO:teuthology.orchestra.run.smithi102.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-16T10:59:54.745 INFO:teuthology.orchestra.run.smithi102.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-16T10:59:54.745 INFO:teuthology.orchestra.run.smithi102.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-16T10:59:54.746 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-16T10:59:56.053 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : ceph-osd-2:17.2.7-1643.gd75f2382.el9.x86_64 31/34 2024-09-16T10:59:56.080 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: ceph-osd-2:17.2.7-1643.gd75f2382.el9.x86_64 31/34 2024-09-16T10:59:56.080 INFO:teuthology.orchestra.run.smithi102.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T10:59:56.080 INFO:teuthology.orchestra.run.smithi102.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-16T10:59:56.080 INFO:teuthology.orchestra.run.smithi102.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-16T10:59:56.080 INFO:teuthology.orchestra.run.smithi102.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-16T10:59:56.080 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-16T10:59:56.462 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : ceph-mon-2:17.2.7-1643.gd75f2382.el9.x86_64 32/34 2024-09-16T10:59:56.486 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: ceph-mon-2:17.2.7-1643.gd75f2382.el9.x86_64 32/34 2024-09-16T10:59:56.486 INFO:teuthology.orchestra.run.smithi102.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T10:59:56.486 INFO:teuthology.orchestra.run.smithi102.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-16T10:59:56.486 INFO:teuthology.orchestra.run.smithi102.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-16T10:59:56.486 INFO:teuthology.orchestra.run.smithi102.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-16T10:59:56.486 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-16T10:59:56.688 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : ceph-mds-2:17.2.7-1643.gd75f2382.el9.x86_64 33/34 2024-09-16T10:59:56.711 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: ceph-mds-2:17.2.7-1643.gd75f2382.el9.x86_64 33/34 2024-09-16T10:59:56.711 INFO:teuthology.orchestra.run.smithi102.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T10:59:56.711 INFO:teuthology.orchestra.run.smithi102.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-16T10:59:56.712 INFO:teuthology.orchestra.run.smithi102.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-16T10:59:56.712 INFO:teuthology.orchestra.run.smithi102.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-16T10:59:56.712 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-16T10:59:56.776 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : ceph-2:17.2.7-1643.gd75f2382.el9.x86_64 34/34 2024-09-16T10:59:58.447 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: ceph-2:17.2.7-1643.gd75f2382.el9.x86_64 34/34 2024-09-16T10:59:58.447 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : ceph-2:17.2.7-1643.gd75f2382.el9.x86_64 1/34 2024-09-16T10:59:58.447 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : ceph-mds-2:17.2.7-1643.gd75f2382.el9.x86_64 2/34 2024-09-16T10:59:58.448 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : ceph-mgr-2:17.2.7-1643.gd75f2382.el9.x86_64 3/34 2024-09-16T10:59:58.448 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : ceph-mon-2:17.2.7-1643.gd75f2382.el9.x86_64 4/34 2024-09-16T10:59:58.448 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : ceph-osd-2:17.2.7-1643.gd75f2382.el9.x86_64 5/34 2024-09-16T10:59:58.448 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : ceph-mgr-modules-core-2:17.2.7-1643.gd75f2382.el9. 6/34 2024-09-16T10:59:58.448 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 7/34 2024-09-16T10:59:58.448 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 8/34 2024-09-16T10:59:58.449 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-ply-3.11-14.el9.noarch 9/34 2024-09-16T10:59:58.449 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 10/34 2024-09-16T10:59:58.449 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 11/34 2024-09-16T10:59:58.449 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 12/34 2024-09-16T10:59:58.449 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 13/34 2024-09-16T10:59:58.449 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 14/34 2024-09-16T10:59:58.450 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 15/34 2024-09-16T10:59:58.450 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 16/34 2024-09-16T10:59:58.450 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 17/34 2024-09-16T10:59:58.450 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 18/34 2024-09-16T10:59:58.450 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 19/34 2024-09-16T10:59:58.450 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 20/34 2024-09-16T10:59:58.451 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 21/34 2024-09-16T10:59:58.451 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 22/34 2024-09-16T10:59:58.451 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 23/34 2024-09-16T10:59:58.451 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 24/34 2024-09-16T10:59:58.451 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 25/34 2024-09-16T10:59:58.451 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 26/34 2024-09-16T10:59:58.451 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 27/34 2024-09-16T10:59:58.452 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 28/34 2024-09-16T10:59:58.452 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 29/34 2024-09-16T10:59:58.452 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 30/34 2024-09-16T10:59:58.452 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 31/34 2024-09-16T10:59:58.452 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 32/34 2024-09-16T10:59:58.453 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 33/34 2024-09-16T10:59:59.267 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 34/34 2024-09-16T10:59:59.267 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-16T10:59:59.267 INFO:teuthology.orchestra.run.smithi102.stdout:Installed: 2024-09-16T10:59:59.267 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-2:17.2.7-1643.gd75f2382.el9.x86_64 2024-09-16T10:59:59.267 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-mds-2:17.2.7-1643.gd75f2382.el9.x86_64 2024-09-16T10:59:59.267 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-mgr-2:17.2.7-1643.gd75f2382.el9.x86_64 2024-09-16T10:59:59.267 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-mgr-modules-core-2:17.2.7-1643.gd75f2382.el9.noarch 2024-09-16T10:59:59.267 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-mon-2:17.2.7-1643.gd75f2382.el9.x86_64 2024-09-16T10:59:59.267 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-osd-2:17.2.7-1643.gd75f2382.el9.x86_64 2024-09-16T10:59:59.268 INFO:teuthology.orchestra.run.smithi102.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-16T10:59:59.268 INFO:teuthology.orchestra.run.smithi102.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-16T10:59:59.268 INFO:teuthology.orchestra.run.smithi102.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-16T10:59:59.268 INFO:teuthology.orchestra.run.smithi102.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-16T10:59:59.268 INFO:teuthology.orchestra.run.smithi102.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-16T10:59:59.268 INFO:teuthology.orchestra.run.smithi102.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-16T10:59:59.268 INFO:teuthology.orchestra.run.smithi102.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-16T10:59:59.268 INFO:teuthology.orchestra.run.smithi102.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-16T10:59:59.268 INFO:teuthology.orchestra.run.smithi102.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-16T10:59:59.268 INFO:teuthology.orchestra.run.smithi102.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-16T10:59:59.268 INFO:teuthology.orchestra.run.smithi102.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-16T10:59:59.268 INFO:teuthology.orchestra.run.smithi102.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-16T10:59:59.268 INFO:teuthology.orchestra.run.smithi102.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-16T10:59:59.269 INFO:teuthology.orchestra.run.smithi102.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-16T10:59:59.269 INFO:teuthology.orchestra.run.smithi102.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-16T10:59:59.269 INFO:teuthology.orchestra.run.smithi102.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-16T10:59:59.269 INFO:teuthology.orchestra.run.smithi102.stdout: python3-ply-3.11-14.el9.noarch 2024-09-16T10:59:59.269 INFO:teuthology.orchestra.run.smithi102.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-16T10:59:59.269 INFO:teuthology.orchestra.run.smithi102.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-16T10:59:59.269 INFO:teuthology.orchestra.run.smithi102.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-16T10:59:59.269 INFO:teuthology.orchestra.run.smithi102.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-16T10:59:59.269 INFO:teuthology.orchestra.run.smithi102.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-16T10:59:59.269 INFO:teuthology.orchestra.run.smithi102.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-16T10:59:59.269 INFO:teuthology.orchestra.run.smithi102.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-16T10:59:59.269 INFO:teuthology.orchestra.run.smithi102.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-16T10:59:59.269 INFO:teuthology.orchestra.run.smithi102.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-16T10:59:59.270 INFO:teuthology.orchestra.run.smithi102.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-16T10:59:59.270 INFO:teuthology.orchestra.run.smithi102.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-16T10:59:59.270 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-16T10:59:59.270 INFO:teuthology.orchestra.run.smithi102.stdout:Complete! 2024-09-16T10:59:59.632 DEBUG:teuthology.orchestra.run.smithi102:> sudo yum -y install ceph-base 2024-09-16T11:00:00.213 INFO:teuthology.orchestra.run.smithi102.stdout:Last metadata expiration check: 0:00:50 ago on Mon 16 Sep 2024 10:59:10 AM UTC. 2024-09-16T11:00:00.301 INFO:teuthology.orchestra.run.smithi102.stdout:Package ceph-base-2:17.2.7-1643.gd75f2382.el9.x86_64 is already installed. 2024-09-16T11:00:00.342 INFO:teuthology.orchestra.run.smithi102.stdout:Dependencies resolved. 2024-09-16T11:00:00.343 INFO:teuthology.orchestra.run.smithi102.stdout:Nothing to do. 2024-09-16T11:00:00.343 INFO:teuthology.orchestra.run.smithi102.stdout:Complete! 2024-09-16T11:00:00.401 DEBUG:teuthology.orchestra.run.smithi102:> sudo yum -y install cephadm 2024-09-16T11:00:00.964 INFO:teuthology.orchestra.run.smithi102.stdout:Last metadata expiration check: 0:00:50 ago on Mon 16 Sep 2024 10:59:10 AM UTC. 2024-09-16T11:00:01.096 INFO:teuthology.orchestra.run.smithi102.stdout:Dependencies resolved. 2024-09-16T11:00:01.097 INFO:teuthology.orchestra.run.smithi102.stdout:================================================================================ 2024-09-16T11:00:01.097 INFO:teuthology.orchestra.run.smithi102.stdout: Package Arch Version Repository Size 2024-09-16T11:00:01.097 INFO:teuthology.orchestra.run.smithi102.stdout:================================================================================ 2024-09-16T11:00:01.097 INFO:teuthology.orchestra.run.smithi102.stdout:Installing: 2024-09-16T11:00:01.097 INFO:teuthology.orchestra.run.smithi102.stdout: cephadm noarch 2:17.2.7-1643.gd75f2382.el9 ceph-noarch 83 k 2024-09-16T11:00:01.098 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-16T11:00:01.098 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction Summary 2024-09-16T11:00:01.098 INFO:teuthology.orchestra.run.smithi102.stdout:================================================================================ 2024-09-16T11:00:01.098 INFO:teuthology.orchestra.run.smithi102.stdout:Install 1 Package 2024-09-16T11:00:01.098 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-16T11:00:01.098 INFO:teuthology.orchestra.run.smithi102.stdout:Total download size: 83 k 2024-09-16T11:00:01.098 INFO:teuthology.orchestra.run.smithi102.stdout:Installed size: 375 k 2024-09-16T11:00:01.098 INFO:teuthology.orchestra.run.smithi102.stdout:Downloading Packages: 2024-09-16T11:00:01.290 INFO:teuthology.orchestra.run.smithi102.stdout:cephadm-17.2.7-1643.gd75f2382.el9.noarch.rpm 432 kB/s | 83 kB 00:00 2024-09-16T11:00:01.291 INFO:teuthology.orchestra.run.smithi102.stdout:-------------------------------------------------------------------------------- 2024-09-16T11:00:01.291 INFO:teuthology.orchestra.run.smithi102.stdout:Total 427 kB/s | 83 kB 00:00 2024-09-16T11:00:01.291 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction check 2024-09-16T11:00:01.296 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction check succeeded. 2024-09-16T11:00:01.296 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction test 2024-09-16T11:00:01.298 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction test succeeded. 2024-09-16T11:00:01.299 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction 2024-09-16T11:00:01.379 INFO:teuthology.orchestra.run.smithi102.stdout: Preparing : 1/1 2024-09-16T11:00:01.872 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: cephadm-2:17.2.7-1643.gd75f2382.el9.noarch 1/1 2024-09-16T11:00:01.961 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : cephadm-2:17.2.7-1643.gd75f2382.el9.noarch 1/1 2024-09-16T11:00:02.287 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: cephadm-2:17.2.7-1643.gd75f2382.el9.noarch 1/1 2024-09-16T11:00:02.535 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : cephadm-2:17.2.7-1643.gd75f2382.el9.noarch 1/1 2024-09-16T11:00:02.536 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-16T11:00:02.536 INFO:teuthology.orchestra.run.smithi102.stdout:Installed: 2024-09-16T11:00:02.536 INFO:teuthology.orchestra.run.smithi102.stdout: cephadm-2:17.2.7-1643.gd75f2382.el9.noarch 2024-09-16T11:00:02.536 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-16T11:00:02.536 INFO:teuthology.orchestra.run.smithi102.stdout:Complete! 2024-09-16T11:00:02.734 DEBUG:teuthology.orchestra.run.smithi102:> sudo yum -y install ceph-immutable-object-cache 2024-09-16T11:00:03.298 INFO:teuthology.orchestra.run.smithi102.stdout:Last metadata expiration check: 0:00:53 ago on Mon 16 Sep 2024 10:59:10 AM UTC. 2024-09-16T11:00:03.432 INFO:teuthology.orchestra.run.smithi102.stdout:Dependencies resolved. 2024-09-16T11:00:03.433 INFO:teuthology.orchestra.run.smithi102.stdout:================================================================================ 2024-09-16T11:00:03.433 INFO:teuthology.orchestra.run.smithi102.stdout: Package Arch Version Repo Size 2024-09-16T11:00:03.433 INFO:teuthology.orchestra.run.smithi102.stdout:================================================================================ 2024-09-16T11:00:03.433 INFO:teuthology.orchestra.run.smithi102.stdout:Installing: 2024-09-16T11:00:03.433 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-immutable-object-cache x86_64 2:17.2.7-1643.gd75f2382.el9 ceph 137 k 2024-09-16T11:00:03.433 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-16T11:00:03.434 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction Summary 2024-09-16T11:00:03.434 INFO:teuthology.orchestra.run.smithi102.stdout:================================================================================ 2024-09-16T11:00:03.434 INFO:teuthology.orchestra.run.smithi102.stdout:Install 1 Package 2024-09-16T11:00:03.434 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-16T11:00:03.434 INFO:teuthology.orchestra.run.smithi102.stdout:Total download size: 137 k 2024-09-16T11:00:03.434 INFO:teuthology.orchestra.run.smithi102.stdout:Installed size: 379 k 2024-09-16T11:00:03.434 INFO:teuthology.orchestra.run.smithi102.stdout:Downloading Packages: 2024-09-16T11:00:03.646 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-immutable-object-cache-17.2.7-1643.gd75f23 649 kB/s | 137 kB 00:00 2024-09-16T11:00:03.646 INFO:teuthology.orchestra.run.smithi102.stdout:-------------------------------------------------------------------------------- 2024-09-16T11:00:03.646 INFO:teuthology.orchestra.run.smithi102.stdout:Total 645 kB/s | 137 kB 00:00 2024-09-16T11:00:03.647 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction check 2024-09-16T11:00:03.654 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction check succeeded. 2024-09-16T11:00:03.654 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction test 2024-09-16T11:00:03.710 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction test succeeded. 2024-09-16T11:00:03.711 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction 2024-09-16T11:00:03.858 INFO:teuthology.orchestra.run.smithi102.stdout: Preparing : 1/1 2024-09-16T11:00:03.913 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : ceph-immutable-object-cache-2:17.2.7-1643.gd75f2382. 1/1 2024-09-16T11:00:03.938 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: ceph-immutable-object-cache-2:17.2.7-1643.gd75f2382. 1/1 2024-09-16T11:00:03.938 INFO:teuthology.orchestra.run.smithi102.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T11:00:03.938 INFO:teuthology.orchestra.run.smithi102.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-16T11:00:03.939 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-16T11:00:04.873 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : ceph-immutable-object-cache-2:17.2.7-1643.gd75f2382. 1/1 2024-09-16T11:00:04.873 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-16T11:00:04.873 INFO:teuthology.orchestra.run.smithi102.stdout:Installed: 2024-09-16T11:00:04.873 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-immutable-object-cache-2:17.2.7-1643.gd75f2382.el9.x86_64 2024-09-16T11:00:04.873 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-16T11:00:04.873 INFO:teuthology.orchestra.run.smithi102.stdout:Complete! 2024-09-16T11:00:05.116 DEBUG:teuthology.orchestra.run.smithi102:> sudo yum -y install ceph-mgr 2024-09-16T11:00:05.681 INFO:teuthology.orchestra.run.smithi102.stdout:Last metadata expiration check: 0:00:55 ago on Mon 16 Sep 2024 10:59:10 AM UTC. 2024-09-16T11:00:05.774 INFO:teuthology.orchestra.run.smithi102.stdout:Package ceph-mgr-2:17.2.7-1643.gd75f2382.el9.x86_64 is already installed. 2024-09-16T11:00:05.816 INFO:teuthology.orchestra.run.smithi102.stdout:Dependencies resolved. 2024-09-16T11:00:05.817 INFO:teuthology.orchestra.run.smithi102.stdout:Nothing to do. 2024-09-16T11:00:05.817 INFO:teuthology.orchestra.run.smithi102.stdout:Complete! 2024-09-16T11:00:05.884 DEBUG:teuthology.orchestra.run.smithi102:> sudo yum -y install ceph-mgr-dashboard 2024-09-16T11:00:06.443 INFO:teuthology.orchestra.run.smithi102.stdout:Last metadata expiration check: 0:00:56 ago on Mon 16 Sep 2024 10:59:10 AM UTC. 2024-09-16T11:00:06.575 INFO:teuthology.orchestra.run.smithi102.stdout:Dependencies resolved. 2024-09-16T11:00:06.576 INFO:teuthology.orchestra.run.smithi102.stdout:================================================================================ 2024-09-16T11:00:06.576 INFO:teuthology.orchestra.run.smithi102.stdout: Package Arch Version Repository Size 2024-09-16T11:00:06.576 INFO:teuthology.orchestra.run.smithi102.stdout:================================================================================ 2024-09-16T11:00:06.576 INFO:teuthology.orchestra.run.smithi102.stdout:Installing: 2024-09-16T11:00:06.576 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-mgr-dashboard noarch 2:17.2.7-1643.gd75f2382.el9 ceph-noarch 3.4 M 2024-09-16T11:00:06.576 INFO:teuthology.orchestra.run.smithi102.stdout:Installing dependencies: 2024-09-16T11:00:06.576 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-grafana-dashboards noarch 2:17.2.7-1643.gd75f2382.el9 ceph-noarch 24 k 2024-09-16T11:00:06.576 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-prometheus-alerts noarch 2:17.2.7-1643.gd75f2382.el9 ceph-noarch 15 k 2024-09-16T11:00:06.577 INFO:teuthology.orchestra.run.smithi102.stdout: python3-jwt noarch 2.4.0-1.el9 epel 41 k 2024-09-16T11:00:06.577 INFO:teuthology.orchestra.run.smithi102.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-16T11:00:06.577 INFO:teuthology.orchestra.run.smithi102.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-16T11:00:06.577 INFO:teuthology.orchestra.run.smithi102.stdout:Installing weak dependencies: 2024-09-16T11:00:06.577 INFO:teuthology.orchestra.run.smithi102.stdout: python3-jwt+crypto noarch 2.4.0-1.el9 epel 9.0 k 2024-09-16T11:00:06.577 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-16T11:00:06.577 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction Summary 2024-09-16T11:00:06.577 INFO:teuthology.orchestra.run.smithi102.stdout:================================================================================ 2024-09-16T11:00:06.577 INFO:teuthology.orchestra.run.smithi102.stdout:Install 7 Packages 2024-09-16T11:00:06.577 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-16T11:00:06.578 INFO:teuthology.orchestra.run.smithi102.stdout:Total download size: 3.7 M 2024-09-16T11:00:06.578 INFO:teuthology.orchestra.run.smithi102.stdout:Installed size: 80 M 2024-09-16T11:00:06.578 INFO:teuthology.orchestra.run.smithi102.stdout:Downloading Packages: 2024-09-16T11:00:07.143 INFO:teuthology.orchestra.run.smithi102.stdout:(1/7): ceph-prometheus-alerts-17.2.7-1643.gd75f 96 kB/s | 15 kB 00:00 2024-09-16T11:00:07.168 INFO:teuthology.orchestra.run.smithi102.stdout:(2/7): ceph-grafana-dashboards-17.2.7-1643.gd75 137 kB/s | 24 kB 00:00 2024-09-16T11:00:07.210 INFO:teuthology.orchestra.run.smithi102.stdout:(3/7): python3-jwt+crypto-2.4.0-1.el9.noarch.rp 135 kB/s | 9.0 kB 00:00 2024-09-16T11:00:07.235 INFO:teuthology.orchestra.run.smithi102.stdout:(4/7): python3-repoze-lru-0.7-16.el9.noarch.rpm 1.3 MB/s | 31 kB 00:00 2024-09-16T11:00:07.261 INFO:teuthology.orchestra.run.smithi102.stdout:(5/7): python3-jwt-2.4.0-1.el9.noarch.rpm 444 kB/s | 41 kB 00:00 2024-09-16T11:00:07.278 INFO:teuthology.orchestra.run.smithi102.stdout:(6/7): python3-routes-2.5.1-5.el9.noarch.rpm 4.4 MB/s | 188 kB 00:00 2024-09-16T11:00:07.344 INFO:teuthology.orchestra.run.smithi102.stdout:(7/7): ceph-mgr-dashboard-17.2.7-1643.gd75f2382 9.5 MB/s | 3.4 MB 00:00 2024-09-16T11:00:07.347 INFO:teuthology.orchestra.run.smithi102.stdout:-------------------------------------------------------------------------------- 2024-09-16T11:00:07.348 INFO:teuthology.orchestra.run.smithi102.stdout:Total 4.7 MB/s | 3.7 MB 00:00 2024-09-16T11:00:07.348 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction check 2024-09-16T11:00:07.356 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction check succeeded. 2024-09-16T11:00:07.356 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction test 2024-09-16T11:00:07.434 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction test succeeded. 2024-09-16T11:00:07.435 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction 2024-09-16T11:00:07.740 INFO:teuthology.orchestra.run.smithi102.stdout: Preparing : 1/1 2024-09-16T11:00:07.796 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-jwt-2.4.0-1.el9.noarch 1/7 2024-09-16T11:00:07.829 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-jwt+crypto-2.4.0-1.el9.noarch 2/7 2024-09-16T11:00:07.884 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 3/7 2024-09-16T11:00:07.917 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 4/7 2024-09-16T11:00:07.945 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : ceph-prometheus-alerts-2:17.2.7-1643.gd75f2382.el9.n 5/7 2024-09-16T11:00:08.935 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : ceph-grafana-dashboards-2:17.2.7-1643.gd75f2382.el9. 6/7 2024-09-16T11:00:08.965 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : ceph-mgr-dashboard-2:17.2.7-1643.gd75f2382.el9.noarc 7/7 2024-09-16T11:00:09.967 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: ceph-mgr-dashboard-2:17.2.7-1643.gd75f2382.el9.noarc 7/7 2024-09-16T11:00:09.968 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : ceph-grafana-dashboards-2:17.2.7-1643.gd75f2382.el9. 1/7 2024-09-16T11:00:09.968 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : ceph-mgr-dashboard-2:17.2.7-1643.gd75f2382.el9.noarc 2/7 2024-09-16T11:00:09.968 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : ceph-prometheus-alerts-2:17.2.7-1643.gd75f2382.el9.n 3/7 2024-09-16T11:00:09.968 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-jwt+crypto-2.4.0-1.el9.noarch 4/7 2024-09-16T11:00:09.968 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-jwt-2.4.0-1.el9.noarch 5/7 2024-09-16T11:00:09.968 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 6/7 2024-09-16T11:00:10.249 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 7/7 2024-09-16T11:00:10.250 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-16T11:00:10.250 INFO:teuthology.orchestra.run.smithi102.stdout:Installed: 2024-09-16T11:00:10.250 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-grafana-dashboards-2:17.2.7-1643.gd75f2382.el9.noarch 2024-09-16T11:00:10.250 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-mgr-dashboard-2:17.2.7-1643.gd75f2382.el9.noarch 2024-09-16T11:00:10.250 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-prometheus-alerts-2:17.2.7-1643.gd75f2382.el9.noarch 2024-09-16T11:00:10.250 INFO:teuthology.orchestra.run.smithi102.stdout: python3-jwt-2.4.0-1.el9.noarch 2024-09-16T11:00:10.250 INFO:teuthology.orchestra.run.smithi102.stdout: python3-jwt+crypto-2.4.0-1.el9.noarch 2024-09-16T11:00:10.250 INFO:teuthology.orchestra.run.smithi102.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-16T11:00:10.250 INFO:teuthology.orchestra.run.smithi102.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-16T11:00:10.250 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-16T11:00:10.250 INFO:teuthology.orchestra.run.smithi102.stdout:Complete! 2024-09-16T11:00:10.449 DEBUG:teuthology.orchestra.run.smithi102:> sudo yum -y install ceph-mgr-diskprediction-local 2024-09-16T11:00:11.011 INFO:teuthology.orchestra.run.smithi102.stdout:Last metadata expiration check: 0:01:01 ago on Mon 16 Sep 2024 10:59:10 AM UTC. 2024-09-16T11:00:11.143 INFO:teuthology.orchestra.run.smithi102.stdout:Dependencies resolved. 2024-09-16T11:00:11.144 INFO:teuthology.orchestra.run.smithi102.stdout:======================================================================================= 2024-09-16T11:00:11.144 INFO:teuthology.orchestra.run.smithi102.stdout: Package Arch Version Repository Size 2024-09-16T11:00:11.144 INFO:teuthology.orchestra.run.smithi102.stdout:======================================================================================= 2024-09-16T11:00:11.144 INFO:teuthology.orchestra.run.smithi102.stdout:Installing: 2024-09-16T11:00:11.144 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-mgr-diskprediction-local noarch 2:17.2.7-1643.gd75f2382.el9 ceph-noarch 7.4 M 2024-09-16T11:00:11.144 INFO:teuthology.orchestra.run.smithi102.stdout:Installing dependencies: 2024-09-16T11:00:11.144 INFO:teuthology.orchestra.run.smithi102.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-16T11:00:11.144 INFO:teuthology.orchestra.run.smithi102.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-16T11:00:11.145 INFO:teuthology.orchestra.run.smithi102.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-16T11:00:11.145 INFO:teuthology.orchestra.run.smithi102.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-16T11:00:11.145 INFO:teuthology.orchestra.run.smithi102.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-16T11:00:11.145 INFO:teuthology.orchestra.run.smithi102.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-16T11:00:11.145 INFO:teuthology.orchestra.run.smithi102.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-16T11:00:11.145 INFO:teuthology.orchestra.run.smithi102.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-16T11:00:11.145 INFO:teuthology.orchestra.run.smithi102.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-16T11:00:11.145 INFO:teuthology.orchestra.run.smithi102.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-16T11:00:11.145 INFO:teuthology.orchestra.run.smithi102.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-16T11:00:11.145 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-16T11:00:11.145 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction Summary 2024-09-16T11:00:11.145 INFO:teuthology.orchestra.run.smithi102.stdout:======================================================================================= 2024-09-16T11:00:11.145 INFO:teuthology.orchestra.run.smithi102.stdout:Install 12 Packages 2024-09-16T11:00:11.146 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-16T11:00:11.146 INFO:teuthology.orchestra.run.smithi102.stdout:Total download size: 38 M 2024-09-16T11:00:11.146 INFO:teuthology.orchestra.run.smithi102.stdout:Installed size: 212 M 2024-09-16T11:00:11.146 INFO:teuthology.orchestra.run.smithi102.stdout:Downloading Packages: 2024-09-16T11:00:11.377 INFO:teuthology.orchestra.run.smithi102.stdout:(1/12): libquadmath-11.5.0-2.el9.x86_64.rpm 1.4 MB/s | 191 kB 00:00 2024-09-16T11:00:11.436 INFO:teuthology.orchestra.run.smithi102.stdout:(2/12): libgfortran-11.5.0-2.el9.x86_64.rpm 4.0 MB/s | 801 kB 00:00 2024-09-16T11:00:11.736 INFO:teuthology.orchestra.run.smithi102.stdout:(3/12): ceph-mgr-diskprediction-local-17.2.7-16 15 MB/s | 7.4 MB 00:00 2024-09-16T11:00:11.753 INFO:teuthology.orchestra.run.smithi102.stdout:(4/12): flexiblas-3.0.4-8.el9.x86_64.rpm 87 kB/s | 33 kB 00:00 2024-09-16T11:00:11.945 INFO:teuthology.orchestra.run.smithi102.stdout:(5/12): openblas-0.3.26-2.el9.x86_64.rpm 201 kB/s | 38 kB 00:00 2024-09-16T11:00:12.020 INFO:teuthology.orchestra.run.smithi102.stdout:(6/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 64 kB/s | 18 kB 00:00 2024-09-16T11:00:12.487 INFO:teuthology.orchestra.run.smithi102.stdout:(7/12): python3-devel-3.9.19-8.el9.x86_64.rpm 524 kB/s | 245 kB 00:00 2024-09-16T11:00:12.554 INFO:teuthology.orchestra.run.smithi102.stdout:(8/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 2.7 MB/s | 3.0 MB 00:01 2024-09-16T11:00:12.671 INFO:teuthology.orchestra.run.smithi102.stdout:(9/12): python3-numpy-f2py-1.20.1-5.el9.x86_64. 2.1 MB/s | 250 kB 00:00 2024-09-16T11:00:12.746 INFO:teuthology.orchestra.run.smithi102.stdout:(10/12): openblas-openmp-0.3.26-2.el9.x86_64.rp 6.2 MB/s | 4.9 MB 00:00 2024-09-16T11:00:13.755 INFO:teuthology.orchestra.run.smithi102.stdout:(11/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 15 MB/s | 16 MB 00:01 2024-09-16T11:00:14.030 INFO:teuthology.orchestra.run.smithi102.stdout:(12/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 3.3 MB/s | 5.1 MB 00:01 2024-09-16T11:00:14.031 INFO:teuthology.orchestra.run.smithi102.stdout:-------------------------------------------------------------------------------- 2024-09-16T11:00:14.031 INFO:teuthology.orchestra.run.smithi102.stdout:Total 13 MB/s | 38 MB 00:02 2024-09-16T11:00:14.386 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction check 2024-09-16T11:00:14.412 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction check succeeded. 2024-09-16T11:00:14.412 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction test 2024-09-16T11:00:14.913 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction test succeeded. 2024-09-16T11:00:14.914 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction 2024-09-16T11:00:15.669 INFO:teuthology.orchestra.run.smithi102.stdout: Preparing : 1/1 2024-09-16T11:00:15.750 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 1/12 2024-09-16T11:00:15.803 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 2/12 2024-09-16T11:00:15.912 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 3/12 2024-09-16T11:00:15.955 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 4/12 2024-09-16T11:00:16.139 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : openblas-0.3.26-2.el9.x86_64 5/12 2024-09-16T11:00:16.170 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 6/12 2024-09-16T11:00:16.248 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2024-09-16T11:00:16.924 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2024-09-16T11:00:17.041 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2024-09-16T11:00:18.285 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2024-09-16T11:00:19.322 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2024-09-16T11:00:19.353 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : ceph-mgr-diskprediction-local-2:17.2.7-1643.gd75f2 12/12 2024-09-16T11:00:21.863 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:17.2.7-1643.gd75f2 12/12 2024-09-16T11:00:21.863 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : ceph-mgr-diskprediction-local-2:17.2.7-1643.gd75f2 1/12 2024-09-16T11:00:21.863 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 2/12 2024-09-16T11:00:21.863 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 3/12 2024-09-16T11:00:21.863 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 4/12 2024-09-16T11:00:21.863 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 5/12 2024-09-16T11:00:21.863 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 6/12 2024-09-16T11:00:21.863 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 7/12 2024-09-16T11:00:21.864 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 8/12 2024-09-16T11:00:21.864 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 9/12 2024-09-16T11:00:21.864 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 10/12 2024-09-16T11:00:21.864 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 11/12 2024-09-16T11:00:22.301 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 12/12 2024-09-16T11:00:22.301 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-16T11:00:22.301 INFO:teuthology.orchestra.run.smithi102.stdout:Installed: 2024-09-16T11:00:22.301 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-mgr-diskprediction-local-2:17.2.7-1643.gd75f2382.el9.noarch 2024-09-16T11:00:22.301 INFO:teuthology.orchestra.run.smithi102.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-16T11:00:22.301 INFO:teuthology.orchestra.run.smithi102.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-16T11:00:22.302 INFO:teuthology.orchestra.run.smithi102.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-16T11:00:22.302 INFO:teuthology.orchestra.run.smithi102.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-16T11:00:22.302 INFO:teuthology.orchestra.run.smithi102.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-16T11:00:22.302 INFO:teuthology.orchestra.run.smithi102.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-16T11:00:22.302 INFO:teuthology.orchestra.run.smithi102.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-16T11:00:22.302 INFO:teuthology.orchestra.run.smithi102.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-16T11:00:22.302 INFO:teuthology.orchestra.run.smithi102.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-16T11:00:22.302 INFO:teuthology.orchestra.run.smithi102.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-16T11:00:22.302 INFO:teuthology.orchestra.run.smithi102.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-16T11:00:22.302 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-16T11:00:22.302 INFO:teuthology.orchestra.run.smithi102.stdout:Complete! 2024-09-16T11:00:22.656 DEBUG:teuthology.orchestra.run.smithi102:> sudo yum -y install ceph-mgr-rook 2024-09-16T11:00:23.241 INFO:teuthology.orchestra.run.smithi102.stdout:Last metadata expiration check: 0:01:13 ago on Mon 16 Sep 2024 10:59:10 AM UTC. 2024-09-16T11:00:23.380 INFO:teuthology.orchestra.run.smithi102.stdout:Dependencies resolved. 2024-09-16T11:00:23.381 INFO:teuthology.orchestra.run.smithi102.stdout:================================================================================ 2024-09-16T11:00:23.381 INFO:teuthology.orchestra.run.smithi102.stdout: Package Arch Version Repository Size 2024-09-16T11:00:23.381 INFO:teuthology.orchestra.run.smithi102.stdout:================================================================================ 2024-09-16T11:00:23.381 INFO:teuthology.orchestra.run.smithi102.stdout:Installing: 2024-09-16T11:00:23.381 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-mgr-rook noarch 2:17.2.7-1643.gd75f2382.el9 ceph-noarch 47 k 2024-09-16T11:00:23.381 INFO:teuthology.orchestra.run.smithi102.stdout:Installing dependencies: 2024-09-16T11:00:23.381 INFO:teuthology.orchestra.run.smithi102.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-16T11:00:23.381 INFO:teuthology.orchestra.run.smithi102.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-16T11:00:23.381 INFO:teuthology.orchestra.run.smithi102.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-16T11:00:23.382 INFO:teuthology.orchestra.run.smithi102.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-16T11:00:23.382 INFO:teuthology.orchestra.run.smithi102.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-16T11:00:23.382 INFO:teuthology.orchestra.run.smithi102.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-16T11:00:23.382 INFO:teuthology.orchestra.run.smithi102.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-16T11:00:23.382 INFO:teuthology.orchestra.run.smithi102.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-16T11:00:23.382 INFO:teuthology.orchestra.run.smithi102.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-16T11:00:23.382 INFO:teuthology.orchestra.run.smithi102.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-16T11:00:23.382 INFO:teuthology.orchestra.run.smithi102.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-16T11:00:23.382 INFO:teuthology.orchestra.run.smithi102.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-16T11:00:23.382 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-16T11:00:23.382 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction Summary 2024-09-16T11:00:23.382 INFO:teuthology.orchestra.run.smithi102.stdout:================================================================================ 2024-09-16T11:00:23.382 INFO:teuthology.orchestra.run.smithi102.stdout:Install 13 Packages 2024-09-16T11:00:23.383 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-16T11:00:23.383 INFO:teuthology.orchestra.run.smithi102.stdout:Total download size: 2.2 M 2024-09-16T11:00:23.383 INFO:teuthology.orchestra.run.smithi102.stdout:Installed size: 26 M 2024-09-16T11:00:23.383 INFO:teuthology.orchestra.run.smithi102.stdout:Downloading Packages: 2024-09-16T11:00:23.770 INFO:teuthology.orchestra.run.smithi102.stdout:(1/13): python3-jsonpatch-1.21-16.el9.noarch.rp 310 kB/s | 26 kB 00:00 2024-09-16T11:00:23.814 INFO:teuthology.orchestra.run.smithi102.stdout:(2/13): python3-jsonpointer-2.0-4.el9.noarch.rp 152 kB/s | 19 kB 00:00 2024-09-16T11:00:23.905 INFO:teuthology.orchestra.run.smithi102.stdout:(3/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 1.7 MB/s | 222 kB 00:00 2024-09-16T11:00:23.929 INFO:teuthology.orchestra.run.smithi102.stdout:(4/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.3 MB/s | 159 kB 00:00 2024-09-16T11:00:23.988 INFO:teuthology.orchestra.run.smithi102.stdout:(5/13): python3-requests-oauthlib-1.3.0-12.el9. 919 kB/s | 54 kB 00:00 2024-09-16T11:00:24.022 INFO:teuthology.orchestra.run.smithi102.stdout:(6/13): ceph-mgr-rook-17.2.7-1643.gd75f2382.el9 142 kB/s | 47 kB 00:00 2024-09-16T11:00:24.047 INFO:teuthology.orchestra.run.smithi102.stdout:(7/13): python3-pyasn1-modules-0.4.8-6.el9.noar 1.9 MB/s | 279 kB 00:00 2024-09-16T11:00:24.081 INFO:teuthology.orchestra.run.smithi102.stdout:(8/13): python3-cachetools-4.2.4-1.el9.noarch.r 349 kB/s | 32 kB 00:00 2024-09-16T11:00:24.114 INFO:teuthology.orchestra.run.smithi102.stdout:(9/13): python3-certifi-2023.05.07-4.el9.noarch 156 kB/s | 14 kB 00:00 2024-09-16T11:00:24.148 INFO:teuthology.orchestra.run.smithi102.stdout:(10/13): python3-rsa-4.9-2.el9.noarch.rpm 1.8 MB/s | 59 kB 00:00 2024-09-16T11:00:24.181 INFO:teuthology.orchestra.run.smithi102.stdout:(11/13): python3-websocket-client-1.2.3-2.el9.n 2.6 MB/s | 90 kB 00:00 2024-09-16T11:00:24.207 INFO:teuthology.orchestra.run.smithi102.stdout:(12/13): python3-google-auth-2.34.0-1.el9.noarc 1.4 MB/s | 220 kB 00:00 2024-09-16T11:00:24.240 INFO:teuthology.orchestra.run.smithi102.stdout:(13/13): python3-kubernetes-26.1.0-2.el9.noarch 6.8 MB/s | 1.0 MB 00:00 2024-09-16T11:00:24.244 INFO:teuthology.orchestra.run.smithi102.stdout:-------------------------------------------------------------------------------- 2024-09-16T11:00:24.244 INFO:teuthology.orchestra.run.smithi102.stdout:Total 2.6 MB/s | 2.2 MB 00:00 2024-09-16T11:00:24.306 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction check 2024-09-16T11:00:24.321 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction check succeeded. 2024-09-16T11:00:24.321 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction test 2024-09-16T11:00:24.407 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction test succeeded. 2024-09-16T11:00:24.407 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction 2024-09-16T11:00:24.923 INFO:teuthology.orchestra.run.smithi102.stdout: Preparing : 1/1 2024-09-16T11:00:25.085 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2024-09-16T11:00:25.149 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2024-09-16T11:00:25.205 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2024-09-16T11:00:25.242 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2024-09-16T11:00:25.284 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 5/13 2024-09-16T11:00:25.384 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2024-09-16T11:00:25.487 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 7/13 2024-09-16T11:00:25.543 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2024-09-16T11:00:26.251 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2024-09-16T11:00:26.316 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2024-09-16T11:00:26.350 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2024-09-16T11:00:26.395 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2024-09-16T11:00:26.420 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : ceph-mgr-rook-2:17.2.7-1643.gd75f2382.el9.noarch 13/13 2024-09-16T11:00:27.177 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: ceph-mgr-rook-2:17.2.7-1643.gd75f2382.el9.noarch 13/13 2024-09-16T11:00:27.178 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : ceph-mgr-rook-2:17.2.7-1643.gd75f2382.el9.noarch 1/13 2024-09-16T11:00:27.178 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2024-09-16T11:00:27.178 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2024-09-16T11:00:27.178 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2024-09-16T11:00:27.178 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2024-09-16T11:00:27.178 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2024-09-16T11:00:27.178 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2024-09-16T11:00:27.178 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2024-09-16T11:00:27.178 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 9/13 2024-09-16T11:00:27.178 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 10/13 2024-09-16T11:00:27.178 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2024-09-16T11:00:27.178 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2024-09-16T11:00:27.523 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2024-09-16T11:00:27.523 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-16T11:00:27.524 INFO:teuthology.orchestra.run.smithi102.stdout:Installed: 2024-09-16T11:00:27.524 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-mgr-rook-2:17.2.7-1643.gd75f2382.el9.noarch 2024-09-16T11:00:27.524 INFO:teuthology.orchestra.run.smithi102.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-16T11:00:27.524 INFO:teuthology.orchestra.run.smithi102.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-16T11:00:27.524 INFO:teuthology.orchestra.run.smithi102.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-16T11:00:27.524 INFO:teuthology.orchestra.run.smithi102.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-16T11:00:27.524 INFO:teuthology.orchestra.run.smithi102.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-16T11:00:27.524 INFO:teuthology.orchestra.run.smithi102.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-16T11:00:27.524 INFO:teuthology.orchestra.run.smithi102.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-16T11:00:27.524 INFO:teuthology.orchestra.run.smithi102.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-16T11:00:27.524 INFO:teuthology.orchestra.run.smithi102.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-16T11:00:27.525 INFO:teuthology.orchestra.run.smithi102.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-16T11:00:27.525 INFO:teuthology.orchestra.run.smithi102.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-16T11:00:27.525 INFO:teuthology.orchestra.run.smithi102.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-16T11:00:27.525 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-16T11:00:27.525 INFO:teuthology.orchestra.run.smithi102.stdout:Complete! 2024-09-16T11:00:27.772 DEBUG:teuthology.orchestra.run.smithi102:> sudo yum -y install ceph-mgr-cephadm 2024-09-16T11:00:28.333 INFO:teuthology.orchestra.run.smithi102.stdout:Last metadata expiration check: 0:01:18 ago on Mon 16 Sep 2024 10:59:10 AM UTC. 2024-09-16T11:00:28.466 INFO:teuthology.orchestra.run.smithi102.stdout:Dependencies resolved. 2024-09-16T11:00:28.467 INFO:teuthology.orchestra.run.smithi102.stdout:================================================================================ 2024-09-16T11:00:28.467 INFO:teuthology.orchestra.run.smithi102.stdout: Package Arch Version Repository Size 2024-09-16T11:00:28.467 INFO:teuthology.orchestra.run.smithi102.stdout:================================================================================ 2024-09-16T11:00:28.467 INFO:teuthology.orchestra.run.smithi102.stdout:Installing: 2024-09-16T11:00:28.467 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-mgr-cephadm noarch 2:17.2.7-1643.gd75f2382.el9 ceph-noarch 124 k 2024-09-16T11:00:28.467 INFO:teuthology.orchestra.run.smithi102.stdout:Installing dependencies: 2024-09-16T11:00:28.467 INFO:teuthology.orchestra.run.smithi102.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-16T11:00:28.467 INFO:teuthology.orchestra.run.smithi102.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-16T11:00:28.467 INFO:teuthology.orchestra.run.smithi102.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-16T11:00:28.468 INFO:teuthology.orchestra.run.smithi102.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-16T11:00:28.468 INFO:teuthology.orchestra.run.smithi102.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-16T11:00:28.468 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-16T11:00:28.468 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction Summary 2024-09-16T11:00:28.468 INFO:teuthology.orchestra.run.smithi102.stdout:================================================================================ 2024-09-16T11:00:28.468 INFO:teuthology.orchestra.run.smithi102.stdout:Install 6 Packages 2024-09-16T11:00:28.468 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-16T11:00:28.469 INFO:teuthology.orchestra.run.smithi102.stdout:Total download size: 7.0 M 2024-09-16T11:00:28.469 INFO:teuthology.orchestra.run.smithi102.stdout:Installed size: 33 M 2024-09-16T11:00:28.469 INFO:teuthology.orchestra.run.smithi102.stdout:Downloading Packages: 2024-09-16T11:00:29.126 INFO:teuthology.orchestra.run.smithi102.stdout:(1/6): python3-jinja2-2.11.3-6.el9.noarch.rpm 1.2 MB/s | 248 kB 00:00 2024-09-16T11:00:29.151 INFO:teuthology.orchestra.run.smithi102.stdout:(2/6): ceph-mgr-cephadm-17.2.7-1643.gd75f2382.e 538 kB/s | 124 kB 00:00 2024-09-16T11:00:29.201 INFO:teuthology.orchestra.run.smithi102.stdout:(3/6): python3-asyncssh-2.13.2-5.el9.noarch.rpm 7.1 MB/s | 548 kB 00:00 2024-09-16T11:00:29.228 INFO:teuthology.orchestra.run.smithi102.stdout:(4/6): python3-natsort-7.1.1-5.el9.noarch.rpm 767 kB/s | 58 kB 00:00 2024-09-16T11:00:29.252 INFO:teuthology.orchestra.run.smithi102.stdout:(5/6): python3-typing-extensions-4.12.2-2.el9.n 1.4 MB/s | 74 kB 00:00 2024-09-16T11:00:31.277 INFO:teuthology.orchestra.run.smithi102.stdout:(6/6): python3-babel-2.9.1-2.el9.noarch.rpm 2.5 MB/s | 6.0 MB 00:02 2024-09-16T11:00:31.280 INFO:teuthology.orchestra.run.smithi102.stdout:-------------------------------------------------------------------------------- 2024-09-16T11:00:31.280 INFO:teuthology.orchestra.run.smithi102.stdout:Total 2.5 MB/s | 7.0 MB 00:02 2024-09-16T11:00:31.353 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction check 2024-09-16T11:00:31.365 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction check succeeded. 2024-09-16T11:00:31.365 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction test 2024-09-16T11:00:31.462 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction test succeeded. 2024-09-16T11:00:31.462 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction 2024-09-16T11:00:31.771 INFO:teuthology.orchestra.run.smithi102.stdout: Preparing : 1/1 2024-09-16T11:00:31.935 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 1/6 2024-09-16T11:00:31.987 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 2/6 2024-09-16T11:00:32.517 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2024-09-16T11:00:32.598 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2024-09-16T11:00:32.653 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 5/6 2024-09-16T11:00:32.682 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : ceph-mgr-cephadm-2:17.2.7-1643.gd75f2382.el9.noarch 6/6 2024-09-16T11:00:33.487 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: ceph-mgr-cephadm-2:17.2.7-1643.gd75f2382.el9.noarch 6/6 2024-09-16T11:00:33.487 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : ceph-mgr-cephadm-2:17.2.7-1643.gd75f2382.el9.noarch 1/6 2024-09-16T11:00:33.488 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2024-09-16T11:00:33.488 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 3/6 2024-09-16T11:00:33.488 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 4/6 2024-09-16T11:00:33.488 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2024-09-16T11:00:33.836 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 6/6 2024-09-16T11:00:33.836 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-16T11:00:33.836 INFO:teuthology.orchestra.run.smithi102.stdout:Installed: 2024-09-16T11:00:33.836 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-mgr-cephadm-2:17.2.7-1643.gd75f2382.el9.noarch 2024-09-16T11:00:33.836 INFO:teuthology.orchestra.run.smithi102.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-16T11:00:33.836 INFO:teuthology.orchestra.run.smithi102.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-16T11:00:33.836 INFO:teuthology.orchestra.run.smithi102.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-16T11:00:33.836 INFO:teuthology.orchestra.run.smithi102.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-16T11:00:33.836 INFO:teuthology.orchestra.run.smithi102.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-16T11:00:33.836 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-16T11:00:33.837 INFO:teuthology.orchestra.run.smithi102.stdout:Complete! 2024-09-16T11:00:34.064 DEBUG:teuthology.orchestra.run.smithi102:> sudo yum -y install ceph-fuse 2024-09-16T11:00:34.635 INFO:teuthology.orchestra.run.smithi102.stdout:Last metadata expiration check: 0:01:24 ago on Mon 16 Sep 2024 10:59:10 AM UTC. 2024-09-16T11:00:34.768 INFO:teuthology.orchestra.run.smithi102.stdout:Dependencies resolved. 2024-09-16T11:00:34.769 INFO:teuthology.orchestra.run.smithi102.stdout:================================================================================ 2024-09-16T11:00:34.769 INFO:teuthology.orchestra.run.smithi102.stdout: Package Arch Version Repository Size 2024-09-16T11:00:34.769 INFO:teuthology.orchestra.run.smithi102.stdout:================================================================================ 2024-09-16T11:00:34.769 INFO:teuthology.orchestra.run.smithi102.stdout:Installing: 2024-09-16T11:00:34.769 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-fuse x86_64 2:17.2.7-1643.gd75f2382.el9 ceph 813 k 2024-09-16T11:00:34.770 INFO:teuthology.orchestra.run.smithi102.stdout:Installing dependencies: 2024-09-16T11:00:34.770 INFO:teuthology.orchestra.run.smithi102.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-16T11:00:34.770 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-16T11:00:34.770 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction Summary 2024-09-16T11:00:34.770 INFO:teuthology.orchestra.run.smithi102.stdout:================================================================================ 2024-09-16T11:00:34.770 INFO:teuthology.orchestra.run.smithi102.stdout:Install 2 Packages 2024-09-16T11:00:34.770 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-16T11:00:34.770 INFO:teuthology.orchestra.run.smithi102.stdout:Total download size: 893 k 2024-09-16T11:00:34.770 INFO:teuthology.orchestra.run.smithi102.stdout:Installed size: 2.6 M 2024-09-16T11:00:34.770 INFO:teuthology.orchestra.run.smithi102.stdout:Downloading Packages: 2024-09-16T11:00:34.948 INFO:teuthology.orchestra.run.smithi102.stdout:(1/2): fuse-2.9.9-16.el9.x86_64.rpm 515 kB/s | 80 kB 00:00 2024-09-16T11:00:35.087 INFO:teuthology.orchestra.run.smithi102.stdout:(2/2): ceph-fuse-17.2.7-1643.gd75f2382.el9.x86_ 2.7 MB/s | 813 kB 00:00 2024-09-16T11:00:35.088 INFO:teuthology.orchestra.run.smithi102.stdout:-------------------------------------------------------------------------------- 2024-09-16T11:00:35.088 INFO:teuthology.orchestra.run.smithi102.stdout:Total 2.7 MB/s | 893 kB 00:00 2024-09-16T11:00:35.099 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction check 2024-09-16T11:00:35.120 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction check succeeded. 2024-09-16T11:00:35.121 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction test 2024-09-16T11:00:35.198 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction test succeeded. 2024-09-16T11:00:35.199 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction 2024-09-16T11:00:35.387 INFO:teuthology.orchestra.run.smithi102.stdout: Preparing : 1/1 2024-09-16T11:00:35.541 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : fuse-2.9.9-16.el9.x86_64 1/2 2024-09-16T11:00:35.610 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : ceph-fuse-2:17.2.7-1643.gd75f2382.el9.x86_64 2/2 2024-09-16T11:00:36.297 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: ceph-fuse-2:17.2.7-1643.gd75f2382.el9.x86_64 2/2 2024-09-16T11:00:36.297 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : ceph-fuse-2:17.2.7-1643.gd75f2382.el9.x86_64 1/2 2024-09-16T11:00:36.570 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 2/2 2024-09-16T11:00:36.570 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-16T11:00:36.570 INFO:teuthology.orchestra.run.smithi102.stdout:Installed: 2024-09-16T11:00:36.570 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-fuse-2:17.2.7-1643.gd75f2382.el9.x86_64 fuse-2.9.9-16.el9.x86_64 2024-09-16T11:00:36.570 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-16T11:00:36.570 INFO:teuthology.orchestra.run.smithi102.stdout:Complete! 2024-09-16T11:00:36.826 DEBUG:teuthology.orchestra.run.smithi102:> sudo yum -y install ceph-volume 2024-09-16T11:00:37.408 INFO:teuthology.orchestra.run.smithi102.stdout:Last metadata expiration check: 0:01:27 ago on Mon 16 Sep 2024 10:59:10 AM UTC. 2024-09-16T11:00:37.542 INFO:teuthology.orchestra.run.smithi102.stdout:Dependencies resolved. 2024-09-16T11:00:37.542 INFO:teuthology.orchestra.run.smithi102.stdout:================================================================================ 2024-09-16T11:00:37.543 INFO:teuthology.orchestra.run.smithi102.stdout: Package Arch Version Repository Size 2024-09-16T11:00:37.543 INFO:teuthology.orchestra.run.smithi102.stdout:================================================================================ 2024-09-16T11:00:37.543 INFO:teuthology.orchestra.run.smithi102.stdout:Installing: 2024-09-16T11:00:37.543 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-volume noarch 2:17.2.7-1643.gd75f2382.el9 ceph-noarch 270 k 2024-09-16T11:00:37.543 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-16T11:00:37.543 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction Summary 2024-09-16T11:00:37.543 INFO:teuthology.orchestra.run.smithi102.stdout:================================================================================ 2024-09-16T11:00:37.543 INFO:teuthology.orchestra.run.smithi102.stdout:Install 1 Package 2024-09-16T11:00:37.543 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-16T11:00:37.543 INFO:teuthology.orchestra.run.smithi102.stdout:Total download size: 270 k 2024-09-16T11:00:37.543 INFO:teuthology.orchestra.run.smithi102.stdout:Installed size: 1.3 M 2024-09-16T11:00:37.543 INFO:teuthology.orchestra.run.smithi102.stdout:Downloading Packages: 2024-09-16T11:00:37.817 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-volume-17.2.7-1643.gd75f2382.el9.noarch.rp 988 kB/s | 270 kB 00:00 2024-09-16T11:00:37.818 INFO:teuthology.orchestra.run.smithi102.stdout:-------------------------------------------------------------------------------- 2024-09-16T11:00:37.818 INFO:teuthology.orchestra.run.smithi102.stdout:Total 983 kB/s | 270 kB 00:00 2024-09-16T11:00:37.818 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction check 2024-09-16T11:00:37.826 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction check succeeded. 2024-09-16T11:00:37.826 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction test 2024-09-16T11:00:37.858 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction test succeeded. 2024-09-16T11:00:37.858 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction 2024-09-16T11:00:38.067 INFO:teuthology.orchestra.run.smithi102.stdout: Preparing : 1/1 2024-09-16T11:00:38.127 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : ceph-volume-2:17.2.7-1643.gd75f2382.el9.noarch 1/1 2024-09-16T11:00:38.144 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: ceph-volume-2:17.2.7-1643.gd75f2382.el9.noarch 1/1 2024-09-16T11:00:38.144 INFO:teuthology.orchestra.run.smithi102.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T11:00:38.144 INFO:teuthology.orchestra.run.smithi102.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-16T11:00:38.144 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-16T11:00:39.138 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : ceph-volume-2:17.2.7-1643.gd75f2382.el9.noarch 1/1 2024-09-16T11:00:39.139 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-16T11:00:39.139 INFO:teuthology.orchestra.run.smithi102.stdout:Installed: 2024-09-16T11:00:39.139 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-volume-2:17.2.7-1643.gd75f2382.el9.noarch 2024-09-16T11:00:39.139 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-16T11:00:39.139 INFO:teuthology.orchestra.run.smithi102.stdout:Complete! 2024-09-16T11:00:39.432 DEBUG:teuthology.orchestra.run.smithi102:> sudo yum -y install librados-devel 2024-09-16T11:00:40.006 INFO:teuthology.orchestra.run.smithi102.stdout:Last metadata expiration check: 0:01:30 ago on Mon 16 Sep 2024 10:59:10 AM UTC. 2024-09-16T11:00:40.140 INFO:teuthology.orchestra.run.smithi102.stdout:Dependencies resolved. 2024-09-16T11:00:40.141 INFO:teuthology.orchestra.run.smithi102.stdout:================================================================================ 2024-09-16T11:00:40.141 INFO:teuthology.orchestra.run.smithi102.stdout: Package Arch Version Repo Size 2024-09-16T11:00:40.141 INFO:teuthology.orchestra.run.smithi102.stdout:================================================================================ 2024-09-16T11:00:40.141 INFO:teuthology.orchestra.run.smithi102.stdout:Installing: 2024-09-16T11:00:40.141 INFO:teuthology.orchestra.run.smithi102.stdout: librados-devel x86_64 2:17.2.7-1643.gd75f2382.el9 ceph 126 k 2024-09-16T11:00:40.141 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-16T11:00:40.141 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction Summary 2024-09-16T11:00:40.141 INFO:teuthology.orchestra.run.smithi102.stdout:================================================================================ 2024-09-16T11:00:40.141 INFO:teuthology.orchestra.run.smithi102.stdout:Install 1 Package 2024-09-16T11:00:40.141 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-16T11:00:40.141 INFO:teuthology.orchestra.run.smithi102.stdout:Total download size: 126 k 2024-09-16T11:00:40.141 INFO:teuthology.orchestra.run.smithi102.stdout:Installed size: 448 k 2024-09-16T11:00:40.142 INFO:teuthology.orchestra.run.smithi102.stdout:Downloading Packages: 2024-09-16T11:00:40.363 INFO:teuthology.orchestra.run.smithi102.stdout:librados-devel-17.2.7-1643.gd75f2382.el9.x86_64 570 kB/s | 126 kB 00:00 2024-09-16T11:00:40.364 INFO:teuthology.orchestra.run.smithi102.stdout:-------------------------------------------------------------------------------- 2024-09-16T11:00:40.364 INFO:teuthology.orchestra.run.smithi102.stdout:Total 565 kB/s | 126 kB 00:00 2024-09-16T11:00:40.364 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction check 2024-09-16T11:00:40.370 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction check succeeded. 2024-09-16T11:00:40.370 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction test 2024-09-16T11:00:40.435 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction test succeeded. 2024-09-16T11:00:40.435 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction 2024-09-16T11:00:40.586 INFO:teuthology.orchestra.run.smithi102.stdout: Preparing : 1/1 2024-09-16T11:00:40.693 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : librados-devel-2:17.2.7-1643.gd75f2382.el9.x86_64 1/1 2024-09-16T11:00:41.105 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: librados-devel-2:17.2.7-1643.gd75f2382.el9.x86_64 1/1 2024-09-16T11:00:41.363 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : librados-devel-2:17.2.7-1643.gd75f2382.el9.x86_64 1/1 2024-09-16T11:00:41.363 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-16T11:00:41.363 INFO:teuthology.orchestra.run.smithi102.stdout:Installed: 2024-09-16T11:00:41.364 INFO:teuthology.orchestra.run.smithi102.stdout: librados-devel-2:17.2.7-1643.gd75f2382.el9.x86_64 2024-09-16T11:00:41.364 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-16T11:00:41.364 INFO:teuthology.orchestra.run.smithi102.stdout:Complete! 2024-09-16T11:00:41.579 DEBUG:teuthology.orchestra.run.smithi102:> sudo yum -y install libcephfs2 2024-09-16T11:00:42.157 INFO:teuthology.orchestra.run.smithi102.stdout:Last metadata expiration check: 0:01:32 ago on Mon 16 Sep 2024 10:59:10 AM UTC. 2024-09-16T11:00:42.249 INFO:teuthology.orchestra.run.smithi102.stdout:Package libcephfs2-2:17.2.7-1643.gd75f2382.el9.x86_64 is already installed. 2024-09-16T11:00:42.294 INFO:teuthology.orchestra.run.smithi102.stdout:Dependencies resolved. 2024-09-16T11:00:42.295 INFO:teuthology.orchestra.run.smithi102.stdout:Nothing to do. 2024-09-16T11:00:42.295 INFO:teuthology.orchestra.run.smithi102.stdout:Complete! 2024-09-16T11:00:42.355 DEBUG:teuthology.orchestra.run.smithi102:> sudo yum -y install libcephfs-devel 2024-09-16T11:00:42.918 INFO:teuthology.orchestra.run.smithi102.stdout:Last metadata expiration check: 0:01:32 ago on Mon 16 Sep 2024 10:59:10 AM UTC. 2024-09-16T11:00:43.049 INFO:teuthology.orchestra.run.smithi102.stdout:Dependencies resolved. 2024-09-16T11:00:43.049 INFO:teuthology.orchestra.run.smithi102.stdout:================================================================================ 2024-09-16T11:00:43.050 INFO:teuthology.orchestra.run.smithi102.stdout: Package Arch Version Repo Size 2024-09-16T11:00:43.050 INFO:teuthology.orchestra.run.smithi102.stdout:================================================================================ 2024-09-16T11:00:43.050 INFO:teuthology.orchestra.run.smithi102.stdout:Installing: 2024-09-16T11:00:43.050 INFO:teuthology.orchestra.run.smithi102.stdout: libcephfs-devel x86_64 2:17.2.7-1643.gd75f2382.el9 ceph 25 k 2024-09-16T11:00:43.050 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-16T11:00:43.050 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction Summary 2024-09-16T11:00:43.050 INFO:teuthology.orchestra.run.smithi102.stdout:================================================================================ 2024-09-16T11:00:43.050 INFO:teuthology.orchestra.run.smithi102.stdout:Install 1 Package 2024-09-16T11:00:43.050 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-16T11:00:43.051 INFO:teuthology.orchestra.run.smithi102.stdout:Total download size: 25 k 2024-09-16T11:00:43.051 INFO:teuthology.orchestra.run.smithi102.stdout:Installed size: 110 k 2024-09-16T11:00:43.051 INFO:teuthology.orchestra.run.smithi102.stdout:Downloading Packages: 2024-09-16T11:00:43.200 INFO:teuthology.orchestra.run.smithi102.stdout:libcephfs-devel-17.2.7-1643.gd75f2382.el9.x86_6 167 kB/s | 25 kB 00:00 2024-09-16T11:00:43.201 INFO:teuthology.orchestra.run.smithi102.stdout:-------------------------------------------------------------------------------- 2024-09-16T11:00:43.201 INFO:teuthology.orchestra.run.smithi102.stdout:Total 166 kB/s | 25 kB 00:00 2024-09-16T11:00:43.201 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction check 2024-09-16T11:00:43.205 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction check succeeded. 2024-09-16T11:00:43.205 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction test 2024-09-16T11:00:43.206 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction test succeeded. 2024-09-16T11:00:43.206 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction 2024-09-16T11:00:43.291 INFO:teuthology.orchestra.run.smithi102.stdout: Preparing : 1/1 2024-09-16T11:00:43.397 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : libcephfs-devel-2:17.2.7-1643.gd75f2382.el9.x86_64 1/1 2024-09-16T11:00:43.786 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: libcephfs-devel-2:17.2.7-1643.gd75f2382.el9.x86_64 1/1 2024-09-16T11:00:44.019 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : libcephfs-devel-2:17.2.7-1643.gd75f2382.el9.x86_64 1/1 2024-09-16T11:00:44.019 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-16T11:00:44.019 INFO:teuthology.orchestra.run.smithi102.stdout:Installed: 2024-09-16T11:00:44.019 INFO:teuthology.orchestra.run.smithi102.stdout: libcephfs-devel-2:17.2.7-1643.gd75f2382.el9.x86_64 2024-09-16T11:00:44.019 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-16T11:00:44.019 INFO:teuthology.orchestra.run.smithi102.stdout:Complete! 2024-09-16T11:00:44.189 DEBUG:teuthology.orchestra.run.smithi102:> sudo yum -y install librados2 2024-09-16T11:00:44.753 INFO:teuthology.orchestra.run.smithi102.stdout:Last metadata expiration check: 0:01:34 ago on Mon 16 Sep 2024 10:59:10 AM UTC. 2024-09-16T11:00:44.843 INFO:teuthology.orchestra.run.smithi102.stdout:Package librados2-2:17.2.7-1643.gd75f2382.el9.x86_64 is already installed. 2024-09-16T11:00:44.886 INFO:teuthology.orchestra.run.smithi102.stdout:Dependencies resolved. 2024-09-16T11:00:44.887 INFO:teuthology.orchestra.run.smithi102.stdout:Nothing to do. 2024-09-16T11:00:44.887 INFO:teuthology.orchestra.run.smithi102.stdout:Complete! 2024-09-16T11:00:44.945 DEBUG:teuthology.orchestra.run.smithi102:> sudo yum -y install librbd1 2024-09-16T11:00:45.500 INFO:teuthology.orchestra.run.smithi102.stdout:Last metadata expiration check: 0:01:35 ago on Mon 16 Sep 2024 10:59:10 AM UTC. 2024-09-16T11:00:45.589 INFO:teuthology.orchestra.run.smithi102.stdout:Package librbd1-2:17.2.7-1643.gd75f2382.el9.x86_64 is already installed. 2024-09-16T11:00:45.632 INFO:teuthology.orchestra.run.smithi102.stdout:Dependencies resolved. 2024-09-16T11:00:45.633 INFO:teuthology.orchestra.run.smithi102.stdout:Nothing to do. 2024-09-16T11:00:45.633 INFO:teuthology.orchestra.run.smithi102.stdout:Complete! 2024-09-16T11:00:45.697 DEBUG:teuthology.orchestra.run.smithi102:> sudo yum -y install python3-rados 2024-09-16T11:00:46.258 INFO:teuthology.orchestra.run.smithi102.stdout:Last metadata expiration check: 0:01:36 ago on Mon 16 Sep 2024 10:59:10 AM UTC. 2024-09-16T11:00:46.347 INFO:teuthology.orchestra.run.smithi102.stdout:Package python3-rados-2:17.2.7-1643.gd75f2382.el9.x86_64 is already installed. 2024-09-16T11:00:46.389 INFO:teuthology.orchestra.run.smithi102.stdout:Dependencies resolved. 2024-09-16T11:00:46.391 INFO:teuthology.orchestra.run.smithi102.stdout:Nothing to do. 2024-09-16T11:00:46.391 INFO:teuthology.orchestra.run.smithi102.stdout:Complete! 2024-09-16T11:00:46.462 DEBUG:teuthology.orchestra.run.smithi102:> sudo yum -y install python3-rgw 2024-09-16T11:00:47.027 INFO:teuthology.orchestra.run.smithi102.stdout:Last metadata expiration check: 0:01:37 ago on Mon 16 Sep 2024 10:59:10 AM UTC. 2024-09-16T11:00:47.117 INFO:teuthology.orchestra.run.smithi102.stdout:Package python3-rgw-2:17.2.7-1643.gd75f2382.el9.x86_64 is already installed. 2024-09-16T11:00:47.160 INFO:teuthology.orchestra.run.smithi102.stdout:Dependencies resolved. 2024-09-16T11:00:47.161 INFO:teuthology.orchestra.run.smithi102.stdout:Nothing to do. 2024-09-16T11:00:47.161 INFO:teuthology.orchestra.run.smithi102.stdout:Complete! 2024-09-16T11:00:47.222 DEBUG:teuthology.orchestra.run.smithi102:> sudo yum -y install python3-cephfs 2024-09-16T11:00:47.781 INFO:teuthology.orchestra.run.smithi102.stdout:Last metadata expiration check: 0:01:37 ago on Mon 16 Sep 2024 10:59:10 AM UTC. 2024-09-16T11:00:47.872 INFO:teuthology.orchestra.run.smithi102.stdout:Package python3-cephfs-2:17.2.7-1643.gd75f2382.el9.x86_64 is already installed. 2024-09-16T11:00:47.914 INFO:teuthology.orchestra.run.smithi102.stdout:Dependencies resolved. 2024-09-16T11:00:47.915 INFO:teuthology.orchestra.run.smithi102.stdout:Nothing to do. 2024-09-16T11:00:47.915 INFO:teuthology.orchestra.run.smithi102.stdout:Complete! 2024-09-16T11:00:47.984 DEBUG:teuthology.orchestra.run.smithi102:> sudo yum -y install python3-rbd 2024-09-16T11:00:48.541 INFO:teuthology.orchestra.run.smithi102.stdout:Last metadata expiration check: 0:01:38 ago on Mon 16 Sep 2024 10:59:10 AM UTC. 2024-09-16T11:00:48.631 INFO:teuthology.orchestra.run.smithi102.stdout:Package python3-rbd-2:17.2.7-1643.gd75f2382.el9.x86_64 is already installed. 2024-09-16T11:00:48.673 INFO:teuthology.orchestra.run.smithi102.stdout:Dependencies resolved. 2024-09-16T11:00:48.674 INFO:teuthology.orchestra.run.smithi102.stdout:Nothing to do. 2024-09-16T11:00:48.674 INFO:teuthology.orchestra.run.smithi102.stdout:Complete! 2024-09-16T11:00:48.738 DEBUG:teuthology.orchestra.run.smithi102:> sudo yum -y install rbd-fuse 2024-09-16T11:00:49.304 INFO:teuthology.orchestra.run.smithi102.stdout:Last metadata expiration check: 0:01:39 ago on Mon 16 Sep 2024 10:59:10 AM UTC. 2024-09-16T11:00:49.437 INFO:teuthology.orchestra.run.smithi102.stdout:Dependencies resolved. 2024-09-16T11:00:49.438 INFO:teuthology.orchestra.run.smithi102.stdout:================================================================================ 2024-09-16T11:00:49.438 INFO:teuthology.orchestra.run.smithi102.stdout: Package Architecture Version Repository Size 2024-09-16T11:00:49.438 INFO:teuthology.orchestra.run.smithi102.stdout:================================================================================ 2024-09-16T11:00:49.438 INFO:teuthology.orchestra.run.smithi102.stdout:Installing: 2024-09-16T11:00:49.438 INFO:teuthology.orchestra.run.smithi102.stdout: rbd-fuse x86_64 2:17.2.7-1643.gd75f2382.el9 ceph 82 k 2024-09-16T11:00:49.438 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-16T11:00:49.438 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction Summary 2024-09-16T11:00:49.438 INFO:teuthology.orchestra.run.smithi102.stdout:================================================================================ 2024-09-16T11:00:49.439 INFO:teuthology.orchestra.run.smithi102.stdout:Install 1 Package 2024-09-16T11:00:49.439 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-16T11:00:49.439 INFO:teuthology.orchestra.run.smithi102.stdout:Total download size: 82 k 2024-09-16T11:00:49.439 INFO:teuthology.orchestra.run.smithi102.stdout:Installed size: 222 k 2024-09-16T11:00:49.439 INFO:teuthology.orchestra.run.smithi102.stdout:Downloading Packages: 2024-09-16T11:00:49.628 INFO:teuthology.orchestra.run.smithi102.stdout:rbd-fuse-17.2.7-1643.gd75f2382.el9.x86_64.rpm 430 kB/s | 82 kB 00:00 2024-09-16T11:00:49.629 INFO:teuthology.orchestra.run.smithi102.stdout:-------------------------------------------------------------------------------- 2024-09-16T11:00:49.629 INFO:teuthology.orchestra.run.smithi102.stdout:Total 427 kB/s | 82 kB 00:00 2024-09-16T11:00:49.629 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction check 2024-09-16T11:00:49.636 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction check succeeded. 2024-09-16T11:00:49.636 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction test 2024-09-16T11:00:49.698 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction test succeeded. 2024-09-16T11:00:49.698 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction 2024-09-16T11:00:49.840 INFO:teuthology.orchestra.run.smithi102.stdout: Preparing : 1/1 2024-09-16T11:00:49.950 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : rbd-fuse-2:17.2.7-1643.gd75f2382.el9.x86_64 1/1 2024-09-16T11:00:50.315 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: rbd-fuse-2:17.2.7-1643.gd75f2382.el9.x86_64 1/1 2024-09-16T11:00:50.562 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : rbd-fuse-2:17.2.7-1643.gd75f2382.el9.x86_64 1/1 2024-09-16T11:00:50.562 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-16T11:00:50.562 INFO:teuthology.orchestra.run.smithi102.stdout:Installed: 2024-09-16T11:00:50.563 INFO:teuthology.orchestra.run.smithi102.stdout: rbd-fuse-2:17.2.7-1643.gd75f2382.el9.x86_64 2024-09-16T11:00:50.563 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-16T11:00:50.563 INFO:teuthology.orchestra.run.smithi102.stdout:Complete! 2024-09-16T11:00:50.787 DEBUG:teuthology.orchestra.run.smithi102:> sudo yum -y install rbd-mirror 2024-09-16T11:00:51.354 INFO:teuthology.orchestra.run.smithi102.stdout:Last metadata expiration check: 0:01:41 ago on Mon 16 Sep 2024 10:59:10 AM UTC. 2024-09-16T11:00:51.486 INFO:teuthology.orchestra.run.smithi102.stdout:Dependencies resolved. 2024-09-16T11:00:51.487 INFO:teuthology.orchestra.run.smithi102.stdout:================================================================================ 2024-09-16T11:00:51.487 INFO:teuthology.orchestra.run.smithi102.stdout: Package Arch Version Repo Size 2024-09-16T11:00:51.487 INFO:teuthology.orchestra.run.smithi102.stdout:================================================================================ 2024-09-16T11:00:51.487 INFO:teuthology.orchestra.run.smithi102.stdout:Installing: 2024-09-16T11:00:51.487 INFO:teuthology.orchestra.run.smithi102.stdout: rbd-mirror x86_64 2:17.2.7-1643.gd75f2382.el9 ceph 2.9 M 2024-09-16T11:00:51.487 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-16T11:00:51.488 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction Summary 2024-09-16T11:00:51.488 INFO:teuthology.orchestra.run.smithi102.stdout:================================================================================ 2024-09-16T11:00:51.488 INFO:teuthology.orchestra.run.smithi102.stdout:Install 1 Package 2024-09-16T11:00:51.488 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-16T11:00:51.488 INFO:teuthology.orchestra.run.smithi102.stdout:Total download size: 2.9 M 2024-09-16T11:00:51.488 INFO:teuthology.orchestra.run.smithi102.stdout:Installed size: 12 M 2024-09-16T11:00:51.488 INFO:teuthology.orchestra.run.smithi102.stdout:Downloading Packages: 2024-09-16T11:00:51.966 INFO:teuthology.orchestra.run.smithi102.stdout:rbd-mirror-17.2.7-1643.gd75f2382.el9.x86_64.rpm 6.2 MB/s | 2.9 MB 00:00 2024-09-16T11:00:51.966 INFO:teuthology.orchestra.run.smithi102.stdout:-------------------------------------------------------------------------------- 2024-09-16T11:00:51.967 INFO:teuthology.orchestra.run.smithi102.stdout:Total 6.1 MB/s | 2.9 MB 00:00 2024-09-16T11:00:51.967 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction check 2024-09-16T11:00:51.975 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction check succeeded. 2024-09-16T11:00:51.975 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction test 2024-09-16T11:00:52.054 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction test succeeded. 2024-09-16T11:00:52.055 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction 2024-09-16T11:00:52.480 INFO:teuthology.orchestra.run.smithi102.stdout: Preparing : 1/1 2024-09-16T11:00:52.650 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : rbd-mirror-2:17.2.7-1643.gd75f2382.el9.x86_64 1/1 2024-09-16T11:00:52.675 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: rbd-mirror-2:17.2.7-1643.gd75f2382.el9.x86_64 1/1 2024-09-16T11:00:52.675 INFO:teuthology.orchestra.run.smithi102.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T11:00:52.675 INFO:teuthology.orchestra.run.smithi102.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-16T11:00:52.675 INFO:teuthology.orchestra.run.smithi102.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-16T11:00:52.675 INFO:teuthology.orchestra.run.smithi102.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-16T11:00:52.675 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-16T11:00:53.615 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : rbd-mirror-2:17.2.7-1643.gd75f2382.el9.x86_64 1/1 2024-09-16T11:00:53.616 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-16T11:00:53.616 INFO:teuthology.orchestra.run.smithi102.stdout:Installed: 2024-09-16T11:00:53.616 INFO:teuthology.orchestra.run.smithi102.stdout: rbd-mirror-2:17.2.7-1643.gd75f2382.el9.x86_64 2024-09-16T11:00:53.616 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-16T11:00:53.616 INFO:teuthology.orchestra.run.smithi102.stdout:Complete! 2024-09-16T11:00:53.861 DEBUG:teuthology.orchestra.run.smithi102:> sudo yum -y install rbd-nbd 2024-09-16T11:00:54.444 INFO:teuthology.orchestra.run.smithi102.stdout:Last metadata expiration check: 0:01:44 ago on Mon 16 Sep 2024 10:59:10 AM UTC. 2024-09-16T11:00:54.585 INFO:teuthology.orchestra.run.smithi102.stdout:Dependencies resolved. 2024-09-16T11:00:54.586 INFO:teuthology.orchestra.run.smithi102.stdout:================================================================================ 2024-09-16T11:00:54.586 INFO:teuthology.orchestra.run.smithi102.stdout: Package Architecture Version Repository Size 2024-09-16T11:00:54.586 INFO:teuthology.orchestra.run.smithi102.stdout:================================================================================ 2024-09-16T11:00:54.586 INFO:teuthology.orchestra.run.smithi102.stdout:Installing: 2024-09-16T11:00:54.586 INFO:teuthology.orchestra.run.smithi102.stdout: rbd-nbd x86_64 2:17.2.7-1643.gd75f2382.el9 ceph 165 k 2024-09-16T11:00:54.586 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-16T11:00:54.587 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction Summary 2024-09-16T11:00:54.587 INFO:teuthology.orchestra.run.smithi102.stdout:================================================================================ 2024-09-16T11:00:54.587 INFO:teuthology.orchestra.run.smithi102.stdout:Install 1 Package 2024-09-16T11:00:54.587 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-16T11:00:54.587 INFO:teuthology.orchestra.run.smithi102.stdout:Total download size: 165 k 2024-09-16T11:00:54.587 INFO:teuthology.orchestra.run.smithi102.stdout:Installed size: 477 k 2024-09-16T11:00:54.587 INFO:teuthology.orchestra.run.smithi102.stdout:Downloading Packages: 2024-09-16T11:00:54.820 INFO:teuthology.orchestra.run.smithi102.stdout:rbd-nbd-17.2.7-1643.gd75f2382.el9.x86_64.rpm 710 kB/s | 165 kB 00:00 2024-09-16T11:00:54.820 INFO:teuthology.orchestra.run.smithi102.stdout:-------------------------------------------------------------------------------- 2024-09-16T11:00:54.821 INFO:teuthology.orchestra.run.smithi102.stdout:Total 704 kB/s | 165 kB 00:00 2024-09-16T11:00:54.821 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction check 2024-09-16T11:00:54.829 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction check succeeded. 2024-09-16T11:00:54.829 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction test 2024-09-16T11:00:54.897 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction test succeeded. 2024-09-16T11:00:54.898 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction 2024-09-16T11:00:55.049 INFO:teuthology.orchestra.run.smithi102.stdout: Preparing : 1/1 2024-09-16T11:00:55.168 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : rbd-nbd-2:17.2.7-1643.gd75f2382.el9.x86_64 1/1 2024-09-16T11:00:55.576 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: rbd-nbd-2:17.2.7-1643.gd75f2382.el9.x86_64 1/1 2024-09-16T11:00:55.828 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : rbd-nbd-2:17.2.7-1643.gd75f2382.el9.x86_64 1/1 2024-09-16T11:00:55.828 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-16T11:00:55.828 INFO:teuthology.orchestra.run.smithi102.stdout:Installed: 2024-09-16T11:00:55.828 INFO:teuthology.orchestra.run.smithi102.stdout: rbd-nbd-2:17.2.7-1643.gd75f2382.el9.x86_64 2024-09-16T11:00:55.828 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-16T11:00:55.829 INFO:teuthology.orchestra.run.smithi102.stdout:Complete! 2024-09-16T11:00:56.071 DEBUG:teuthology.orchestra.run.smithi102:> sudo yum -y install ceph-base-debuginfo 2024-09-16T11:00:56.654 INFO:teuthology.orchestra.run.smithi102.stdout:Last metadata expiration check: 0:01:46 ago on Mon 16 Sep 2024 10:59:10 AM UTC. 2024-09-16T11:00:56.790 INFO:teuthology.orchestra.run.smithi102.stdout:Dependencies resolved. 2024-09-16T11:00:56.790 INFO:teuthology.orchestra.run.smithi102.stdout:================================================================================ 2024-09-16T11:00:56.790 INFO:teuthology.orchestra.run.smithi102.stdout: Package Arch Version Repo Size 2024-09-16T11:00:56.790 INFO:teuthology.orchestra.run.smithi102.stdout:================================================================================ 2024-09-16T11:00:56.790 INFO:teuthology.orchestra.run.smithi102.stdout:Installing: 2024-09-16T11:00:56.791 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-base-debuginfo x86_64 2:17.2.7-1643.gd75f2382.el9 ceph 46 M 2024-09-16T11:00:56.791 INFO:teuthology.orchestra.run.smithi102.stdout:Installing dependencies: 2024-09-16T11:00:56.791 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-debuginfo x86_64 2:17.2.7-1643.gd75f2382.el9 ceph 32 M 2024-09-16T11:00:56.791 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-16T11:00:56.791 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction Summary 2024-09-16T11:00:56.791 INFO:teuthology.orchestra.run.smithi102.stdout:================================================================================ 2024-09-16T11:00:56.791 INFO:teuthology.orchestra.run.smithi102.stdout:Install 2 Packages 2024-09-16T11:00:56.791 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-16T11:00:56.792 INFO:teuthology.orchestra.run.smithi102.stdout:Total download size: 78 M 2024-09-16T11:00:56.792 INFO:teuthology.orchestra.run.smithi102.stdout:Installed size: 430 M 2024-09-16T11:00:56.792 INFO:teuthology.orchestra.run.smithi102.stdout:Downloading Packages: 2024-09-16T11:00:58.832 INFO:teuthology.orchestra.run.smithi102.stdout:(1/2): ceph-base-debuginfo-17.2.7-1643.gd75f238 22 MB/s | 46 MB 00:02 2024-09-16T11:00:59.824 INFO:teuthology.orchestra.run.smithi102.stdout:(2/2): ceph-debuginfo-17.2.7-1643.gd75f2382.el9 10 MB/s | 32 MB 00:03 2024-09-16T11:00:59.826 INFO:teuthology.orchestra.run.smithi102.stdout:-------------------------------------------------------------------------------- 2024-09-16T11:00:59.826 INFO:teuthology.orchestra.run.smithi102.stdout:Total 26 MB/s | 78 MB 00:03 2024-09-16T11:00:59.826 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction check 2024-09-16T11:00:59.831 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction check succeeded. 2024-09-16T11:00:59.831 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction test 2024-09-16T11:01:00.335 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction test succeeded. 2024-09-16T11:01:00.335 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction 2024-09-16T11:01:04.072 INFO:teuthology.orchestra.run.smithi102.stdout: Preparing : 1/1 2024-09-16T11:01:09.162 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : ceph-debuginfo-2:17.2.7-1643.gd75f2382.el9.x86_64 1/2 2024-09-16T11:01:09.233 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : ceph-base-debuginfo-2:17.2.7-1643.gd75f2382.el9.x86_ 2/2 2024-09-16T11:01:12.220 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: ceph-base-debuginfo-2:17.2.7-1643.gd75f2382.el9.x86_ 2/2 2024-09-16T11:01:12.221 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : ceph-base-debuginfo-2:17.2.7-1643.gd75f2382.el9.x86_ 1/2 2024-09-16T11:01:12.450 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : ceph-debuginfo-2:17.2.7-1643.gd75f2382.el9.x86_64 2/2 2024-09-16T11:01:12.450 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-16T11:01:12.450 INFO:teuthology.orchestra.run.smithi102.stdout:Installed: 2024-09-16T11:01:12.450 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-base-debuginfo-2:17.2.7-1643.gd75f2382.el9.x86_64 2024-09-16T11:01:12.450 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-debuginfo-2:17.2.7-1643.gd75f2382.el9.x86_64 2024-09-16T11:01:12.450 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-16T11:01:12.450 INFO:teuthology.orchestra.run.smithi102.stdout:Complete! 2024-09-16T11:01:12.652 DEBUG:teuthology.orchestra.run.smithi102:> sudo yum -y install ceph-common-debuginfo 2024-09-16T11:01:13.213 INFO:teuthology.orchestra.run.smithi102.stdout:Last metadata expiration check: 0:02:03 ago on Mon 16 Sep 2024 10:59:10 AM UTC. 2024-09-16T11:01:13.345 INFO:teuthology.orchestra.run.smithi102.stdout:Dependencies resolved. 2024-09-16T11:01:13.346 INFO:teuthology.orchestra.run.smithi102.stdout:================================================================================ 2024-09-16T11:01:13.346 INFO:teuthology.orchestra.run.smithi102.stdout: Package Arch Version Repo Size 2024-09-16T11:01:13.346 INFO:teuthology.orchestra.run.smithi102.stdout:================================================================================ 2024-09-16T11:01:13.347 INFO:teuthology.orchestra.run.smithi102.stdout:Installing: 2024-09-16T11:01:13.347 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-common-debuginfo x86_64 2:17.2.7-1643.gd75f2382.el9 ceph 441 M 2024-09-16T11:01:13.347 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-16T11:01:13.347 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction Summary 2024-09-16T11:01:13.347 INFO:teuthology.orchestra.run.smithi102.stdout:================================================================================ 2024-09-16T11:01:13.348 INFO:teuthology.orchestra.run.smithi102.stdout:Install 1 Package 2024-09-16T11:01:13.348 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-16T11:01:13.348 INFO:teuthology.orchestra.run.smithi102.stdout:Total download size: 441 M 2024-09-16T11:01:13.348 INFO:teuthology.orchestra.run.smithi102.stdout:Installed size: 1.9 G 2024-09-16T11:01:13.348 INFO:teuthology.orchestra.run.smithi102.stdout:Downloading Packages: 2024-09-16T11:01:33.746 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-common-debuginfo-17.2.7-1643.gd75f2382.el9 22 MB/s | 441 MB 00:20 2024-09-16T11:01:33.746 INFO:teuthology.orchestra.run.smithi102.stdout:-------------------------------------------------------------------------------- 2024-09-16T11:01:33.747 INFO:teuthology.orchestra.run.smithi102.stdout:Total 22 MB/s | 441 MB 00:20 2024-09-16T11:01:33.747 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction check 2024-09-16T11:01:33.751 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction check succeeded. 2024-09-16T11:01:33.751 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction test 2024-09-16T11:01:36.370 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction test succeeded. 2024-09-16T11:01:36.370 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction 2024-09-16T11:02:22.331 INFO:teuthology.orchestra.run.smithi102.stdout: Preparing : 1/1 2024-09-16T11:02:22.676 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : ceph-common-debuginfo-2:17.2.7-1643.gd75f2382.el9.x8 1/1 2024-09-16T11:02:26.645 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: ceph-common-debuginfo-2:17.2.7-1643.gd75f2382.el9.x8 1/1 2024-09-16T11:02:26.864 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : ceph-common-debuginfo-2:17.2.7-1643.gd75f2382.el9.x8 1/1 2024-09-16T11:02:26.864 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-16T11:02:26.864 INFO:teuthology.orchestra.run.smithi102.stdout:Installed: 2024-09-16T11:02:26.864 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-common-debuginfo-2:17.2.7-1643.gd75f2382.el9.x86_64 2024-09-16T11:02:26.864 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-16T11:02:26.864 INFO:teuthology.orchestra.run.smithi102.stdout:Complete! 2024-09-16T11:02:27.071 DEBUG:teuthology.orchestra.run.smithi102:> sudo yum -y install ceph-immutable-object-cache-debuginfo 2024-09-16T11:02:27.640 INFO:teuthology.orchestra.run.smithi102.stdout:Last metadata expiration check: 0:03:17 ago on Mon 16 Sep 2024 10:59:10 AM UTC. 2024-09-16T11:02:27.777 INFO:teuthology.orchestra.run.smithi102.stdout:Dependencies resolved. 2024-09-16T11:02:27.778 INFO:teuthology.orchestra.run.smithi102.stdout:======================================================================================== 2024-09-16T11:02:27.778 INFO:teuthology.orchestra.run.smithi102.stdout: Package Arch Version Repo Size 2024-09-16T11:02:27.778 INFO:teuthology.orchestra.run.smithi102.stdout:======================================================================================== 2024-09-16T11:02:27.778 INFO:teuthology.orchestra.run.smithi102.stdout:Installing: 2024-09-16T11:02:27.778 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-immutable-object-cache-debuginfo x86_64 2:17.2.7-1643.gd75f2382.el9 ceph 1.5 M 2024-09-16T11:02:27.778 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-16T11:02:27.778 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction Summary 2024-09-16T11:02:27.778 INFO:teuthology.orchestra.run.smithi102.stdout:======================================================================================== 2024-09-16T11:02:27.778 INFO:teuthology.orchestra.run.smithi102.stdout:Install 1 Package 2024-09-16T11:02:27.778 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-16T11:02:27.778 INFO:teuthology.orchestra.run.smithi102.stdout:Total download size: 1.5 M 2024-09-16T11:02:27.779 INFO:teuthology.orchestra.run.smithi102.stdout:Installed size: 6.5 M 2024-09-16T11:02:27.779 INFO:teuthology.orchestra.run.smithi102.stdout:Downloading Packages: 2024-09-16T11:02:28.184 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-immutable-object-cache-debuginfo-17.2.7-16 3.6 MB/s | 1.5 MB 00:00 2024-09-16T11:02:28.185 INFO:teuthology.orchestra.run.smithi102.stdout:-------------------------------------------------------------------------------- 2024-09-16T11:02:28.185 INFO:teuthology.orchestra.run.smithi102.stdout:Total 3.6 MB/s | 1.5 MB 00:00 2024-09-16T11:02:28.185 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction check 2024-09-16T11:02:28.189 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction check succeeded. 2024-09-16T11:02:28.189 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction test 2024-09-16T11:02:28.234 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction test succeeded. 2024-09-16T11:02:28.234 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction 2024-09-16T11:02:28.500 INFO:teuthology.orchestra.run.smithi102.stdout: Preparing : 1/1 2024-09-16T11:02:28.595 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : ceph-immutable-object-cache-debuginfo-2:17.2.7-1643. 1/1 2024-09-16T11:02:28.991 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: ceph-immutable-object-cache-debuginfo-2:17.2.7-1643. 1/1 2024-09-16T11:02:29.213 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : ceph-immutable-object-cache-debuginfo-2:17.2.7-1643. 1/1 2024-09-16T11:02:29.213 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-16T11:02:29.213 INFO:teuthology.orchestra.run.smithi102.stdout:Installed: 2024-09-16T11:02:29.213 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-immutable-object-cache-debuginfo-2:17.2.7-1643.gd75f2382.el9.x86_64 2024-09-16T11:02:29.213 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-16T11:02:29.214 INFO:teuthology.orchestra.run.smithi102.stdout:Complete! 2024-09-16T11:02:29.359 DEBUG:teuthology.orchestra.run.smithi102:> sudo yum -y install ceph-radosgw-debuginfo 2024-09-16T11:02:29.924 INFO:teuthology.orchestra.run.smithi102.stdout:Last metadata expiration check: 0:03:19 ago on Mon 16 Sep 2024 10:59:10 AM UTC. 2024-09-16T11:02:30.054 INFO:teuthology.orchestra.run.smithi102.stdout:Dependencies resolved. 2024-09-16T11:02:30.055 INFO:teuthology.orchestra.run.smithi102.stdout:================================================================================ 2024-09-16T11:02:30.055 INFO:teuthology.orchestra.run.smithi102.stdout: Package Arch Version Repo Size 2024-09-16T11:02:30.055 INFO:teuthology.orchestra.run.smithi102.stdout:================================================================================ 2024-09-16T11:02:30.055 INFO:teuthology.orchestra.run.smithi102.stdout:Installing: 2024-09-16T11:02:30.055 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-radosgw-debuginfo x86_64 2:17.2.7-1643.gd75f2382.el9 ceph 245 M 2024-09-16T11:02:30.055 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-16T11:02:30.055 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction Summary 2024-09-16T11:02:30.055 INFO:teuthology.orchestra.run.smithi102.stdout:================================================================================ 2024-09-16T11:02:30.055 INFO:teuthology.orchestra.run.smithi102.stdout:Install 1 Package 2024-09-16T11:02:30.056 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-16T11:02:30.056 INFO:teuthology.orchestra.run.smithi102.stdout:Total download size: 245 M 2024-09-16T11:02:30.056 INFO:teuthology.orchestra.run.smithi102.stdout:Installed size: 1.0 G 2024-09-16T11:02:30.056 INFO:teuthology.orchestra.run.smithi102.stdout:Downloading Packages: 2024-09-16T11:02:40.891 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-radosgw-debuginfo-17.2.7-1643.gd75f2382.el 23 MB/s | 245 MB 00:10 2024-09-16T11:02:40.891 INFO:teuthology.orchestra.run.smithi102.stdout:-------------------------------------------------------------------------------- 2024-09-16T11:02:40.892 INFO:teuthology.orchestra.run.smithi102.stdout:Total 23 MB/s | 245 MB 00:10 2024-09-16T11:02:40.892 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction check 2024-09-16T11:02:40.896 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction check succeeded. 2024-09-16T11:02:40.896 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction test 2024-09-16T11:02:42.372 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction test succeeded. 2024-09-16T11:02:42.372 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction 2024-09-16T11:03:07.435 INFO:teuthology.orchestra.run.smithi102.stdout: Preparing : 1/1 2024-09-16T11:03:07.535 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : ceph-radosgw-debuginfo-2:17.2.7-1643.gd75f2382.el9.x 1/1 2024-09-16T11:03:13.804 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: ceph-radosgw-debuginfo-2:17.2.7-1643.gd75f2382.el9.x 1/1 2024-09-16T11:03:14.040 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : ceph-radosgw-debuginfo-2:17.2.7-1643.gd75f2382.el9.x 1/1 2024-09-16T11:03:14.040 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-16T11:03:14.041 INFO:teuthology.orchestra.run.smithi102.stdout:Installed: 2024-09-16T11:03:14.041 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-radosgw-debuginfo-2:17.2.7-1643.gd75f2382.el9.x86_64 2024-09-16T11:03:14.041 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-16T11:03:14.041 INFO:teuthology.orchestra.run.smithi102.stdout:Complete! 2024-09-16T11:03:14.294 DEBUG:teuthology.orchestra.run.smithi102:> sudo yum -y install ceph-test-debuginfo 2024-09-16T11:03:14.888 INFO:teuthology.orchestra.run.smithi102.stdout:Last metadata expiration check: 0:04:04 ago on Mon 16 Sep 2024 10:59:10 AM UTC. 2024-09-16T11:03:15.025 INFO:teuthology.orchestra.run.smithi102.stdout:Dependencies resolved. 2024-09-16T11:03:15.026 INFO:teuthology.orchestra.run.smithi102.stdout:================================================================================ 2024-09-16T11:03:15.027 INFO:teuthology.orchestra.run.smithi102.stdout: Package Arch Version Repo Size 2024-09-16T11:03:15.027 INFO:teuthology.orchestra.run.smithi102.stdout:================================================================================ 2024-09-16T11:03:15.027 INFO:teuthology.orchestra.run.smithi102.stdout:Installing: 2024-09-16T11:03:15.027 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-test-debuginfo x86_64 2:17.2.7-1643.gd75f2382.el9 ceph 627 M 2024-09-16T11:03:15.027 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-16T11:03:15.028 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction Summary 2024-09-16T11:03:15.028 INFO:teuthology.orchestra.run.smithi102.stdout:================================================================================ 2024-09-16T11:03:15.028 INFO:teuthology.orchestra.run.smithi102.stdout:Install 1 Package 2024-09-16T11:03:15.028 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-16T11:03:15.028 INFO:teuthology.orchestra.run.smithi102.stdout:Total download size: 627 M 2024-09-16T11:03:15.028 INFO:teuthology.orchestra.run.smithi102.stdout:Installed size: 2.9 G 2024-09-16T11:03:15.029 INFO:teuthology.orchestra.run.smithi102.stdout:Downloading Packages: 2024-09-16T11:03:53.221 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-test-debuginfo-17.2.7-1643.gd75f2382.el9.x 16 MB/s | 627 MB 00:38 2024-09-16T11:03:53.222 INFO:teuthology.orchestra.run.smithi102.stdout:-------------------------------------------------------------------------------- 2024-09-16T11:03:53.222 INFO:teuthology.orchestra.run.smithi102.stdout:Total 16 MB/s | 627 MB 00:38 2024-09-16T11:03:53.223 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction check 2024-09-16T11:03:53.228 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction check succeeded. 2024-09-16T11:03:53.228 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction test 2024-09-16T11:03:56.930 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction test succeeded. 2024-09-16T11:03:56.930 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction 2024-09-16T11:05:02.852 INFO:teuthology.orchestra.run.smithi102.stdout: Preparing : 1/1 2024-09-16T11:05:03.375 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : ceph-test-debuginfo-2:17.2.7-1643.gd75f2382.el9.x86_ 1/1 2024-09-16T11:05:13.111 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: ceph-test-debuginfo-2:17.2.7-1643.gd75f2382.el9.x86_ 1/1 2024-09-16T11:05:13.340 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : ceph-test-debuginfo-2:17.2.7-1643.gd75f2382.el9.x86_ 1/1 2024-09-16T11:05:13.340 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-16T11:05:13.340 INFO:teuthology.orchestra.run.smithi102.stdout:Installed: 2024-09-16T11:05:13.340 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-test-debuginfo-2:17.2.7-1643.gd75f2382.el9.x86_64 2024-09-16T11:05:13.340 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-16T11:05:13.340 INFO:teuthology.orchestra.run.smithi102.stdout:Complete! 2024-09-16T11:05:13.596 DEBUG:teuthology.orchestra.run.smithi102:> sudo yum -y install ceph-base-debuginfo 2024-09-16T11:05:14.162 INFO:teuthology.orchestra.run.smithi102.stdout:Last metadata expiration check: 0:06:04 ago on Mon 16 Sep 2024 10:59:10 AM UTC. 2024-09-16T11:05:14.253 INFO:teuthology.orchestra.run.smithi102.stdout:Package ceph-base-debuginfo-2:17.2.7-1643.gd75f2382.el9.x86_64 is already installed. 2024-09-16T11:05:14.295 INFO:teuthology.orchestra.run.smithi102.stdout:Dependencies resolved. 2024-09-16T11:05:14.296 INFO:teuthology.orchestra.run.smithi102.stdout:Nothing to do. 2024-09-16T11:05:14.296 INFO:teuthology.orchestra.run.smithi102.stdout:Complete! 2024-09-16T11:05:14.365 DEBUG:teuthology.orchestra.run.smithi102:> sudo yum -y install ceph-mgr-debuginfo 2024-09-16T11:05:14.934 INFO:teuthology.orchestra.run.smithi102.stdout:Last metadata expiration check: 0:06:04 ago on Mon 16 Sep 2024 10:59:10 AM UTC. 2024-09-16T11:05:15.067 INFO:teuthology.orchestra.run.smithi102.stdout:Dependencies resolved. 2024-09-16T11:05:15.069 INFO:teuthology.orchestra.run.smithi102.stdout:================================================================================ 2024-09-16T11:05:15.069 INFO:teuthology.orchestra.run.smithi102.stdout: Package Arch Version Repo Size 2024-09-16T11:05:15.069 INFO:teuthology.orchestra.run.smithi102.stdout:================================================================================ 2024-09-16T11:05:15.069 INFO:teuthology.orchestra.run.smithi102.stdout:Installing: 2024-09-16T11:05:15.069 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-mgr-debuginfo x86_64 2:17.2.7-1643.gd75f2382.el9 ceph 20 M 2024-09-16T11:05:15.069 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-16T11:05:15.069 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction Summary 2024-09-16T11:05:15.069 INFO:teuthology.orchestra.run.smithi102.stdout:================================================================================ 2024-09-16T11:05:15.069 INFO:teuthology.orchestra.run.smithi102.stdout:Install 1 Package 2024-09-16T11:05:15.070 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-16T11:05:15.070 INFO:teuthology.orchestra.run.smithi102.stdout:Total download size: 20 M 2024-09-16T11:05:15.070 INFO:teuthology.orchestra.run.smithi102.stdout:Installed size: 123 M 2024-09-16T11:05:15.070 INFO:teuthology.orchestra.run.smithi102.stdout:Downloading Packages: 2024-09-16T11:05:16.020 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mgr-debuginfo-17.2.7-1643.gd75f2382.el9.x8 21 MB/s | 20 MB 00:00 2024-09-16T11:05:16.021 INFO:teuthology.orchestra.run.smithi102.stdout:-------------------------------------------------------------------------------- 2024-09-16T11:05:16.022 INFO:teuthology.orchestra.run.smithi102.stdout:Total 21 MB/s | 20 MB 00:00 2024-09-16T11:05:16.022 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction check 2024-09-16T11:05:16.026 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction check succeeded. 2024-09-16T11:05:16.026 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction test 2024-09-16T11:05:16.179 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction test succeeded. 2024-09-16T11:05:16.180 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction 2024-09-16T11:05:18.594 INFO:teuthology.orchestra.run.smithi102.stdout: Preparing : 1/1 2024-09-16T11:05:18.707 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : ceph-mgr-debuginfo-2:17.2.7-1643.gd75f2382.el9.x86_6 1/1 2024-09-16T11:05:19.771 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: ceph-mgr-debuginfo-2:17.2.7-1643.gd75f2382.el9.x86_6 1/1 2024-09-16T11:05:19.998 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : ceph-mgr-debuginfo-2:17.2.7-1643.gd75f2382.el9.x86_6 1/1 2024-09-16T11:05:19.998 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-16T11:05:19.999 INFO:teuthology.orchestra.run.smithi102.stdout:Installed: 2024-09-16T11:05:19.999 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-mgr-debuginfo-2:17.2.7-1643.gd75f2382.el9.x86_64 2024-09-16T11:05:19.999 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-16T11:05:19.999 INFO:teuthology.orchestra.run.smithi102.stdout:Complete! 2024-09-16T11:05:20.167 DEBUG:teuthology.orchestra.run.smithi102:> sudo yum -y install ceph-mds-debuginfo 2024-09-16T11:05:20.733 INFO:teuthology.orchestra.run.smithi102.stdout:Last metadata expiration check: 0:06:10 ago on Mon 16 Sep 2024 10:59:10 AM UTC. 2024-09-16T11:05:20.866 INFO:teuthology.orchestra.run.smithi102.stdout:Dependencies resolved. 2024-09-16T11:05:20.867 INFO:teuthology.orchestra.run.smithi102.stdout:================================================================================ 2024-09-16T11:05:20.867 INFO:teuthology.orchestra.run.smithi102.stdout: Package Arch Version Repo Size 2024-09-16T11:05:20.867 INFO:teuthology.orchestra.run.smithi102.stdout:================================================================================ 2024-09-16T11:05:20.867 INFO:teuthology.orchestra.run.smithi102.stdout:Installing: 2024-09-16T11:05:20.867 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-mds-debuginfo x86_64 2:17.2.7-1643.gd75f2382.el9 ceph 49 M 2024-09-16T11:05:20.867 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-16T11:05:20.868 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction Summary 2024-09-16T11:05:20.868 INFO:teuthology.orchestra.run.smithi102.stdout:================================================================================ 2024-09-16T11:05:20.868 INFO:teuthology.orchestra.run.smithi102.stdout:Install 1 Package 2024-09-16T11:05:20.868 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-16T11:05:20.868 INFO:teuthology.orchestra.run.smithi102.stdout:Total download size: 49 M 2024-09-16T11:05:20.868 INFO:teuthology.orchestra.run.smithi102.stdout:Installed size: 205 M 2024-09-16T11:05:20.868 INFO:teuthology.orchestra.run.smithi102.stdout:Downloading Packages: 2024-09-16T11:05:24.140 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mds-debuginfo-17.2.7-1643.gd75f2382.el9.x8 15 MB/s | 49 MB 00:03 2024-09-16T11:05:24.140 INFO:teuthology.orchestra.run.smithi102.stdout:-------------------------------------------------------------------------------- 2024-09-16T11:05:24.141 INFO:teuthology.orchestra.run.smithi102.stdout:Total 15 MB/s | 49 MB 00:03 2024-09-16T11:05:24.141 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction check 2024-09-16T11:05:24.145 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction check succeeded. 2024-09-16T11:05:24.145 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction test 2024-09-16T11:05:24.467 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction test succeeded. 2024-09-16T11:05:24.467 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction 2024-09-16T11:05:29.499 INFO:teuthology.orchestra.run.smithi102.stdout: Preparing : 1/1 2024-09-16T11:05:29.586 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : ceph-mds-debuginfo-2:17.2.7-1643.gd75f2382.el9.x86_6 1/1 2024-09-16T11:05:31.149 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: ceph-mds-debuginfo-2:17.2.7-1643.gd75f2382.el9.x86_6 1/1 2024-09-16T11:05:31.374 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : ceph-mds-debuginfo-2:17.2.7-1643.gd75f2382.el9.x86_6 1/1 2024-09-16T11:05:31.374 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-16T11:05:31.374 INFO:teuthology.orchestra.run.smithi102.stdout:Installed: 2024-09-16T11:05:31.374 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-mds-debuginfo-2:17.2.7-1643.gd75f2382.el9.x86_64 2024-09-16T11:05:31.375 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-16T11:05:31.375 INFO:teuthology.orchestra.run.smithi102.stdout:Complete! 2024-09-16T11:05:31.534 DEBUG:teuthology.orchestra.run.smithi102:> sudo yum -y install ceph-mon-debuginfo 2024-09-16T11:05:32.096 INFO:teuthology.orchestra.run.smithi102.stdout:Last metadata expiration check: 0:06:22 ago on Mon 16 Sep 2024 10:59:10 AM UTC. 2024-09-16T11:05:32.229 INFO:teuthology.orchestra.run.smithi102.stdout:Dependencies resolved. 2024-09-16T11:05:32.229 INFO:teuthology.orchestra.run.smithi102.stdout:================================================================================ 2024-09-16T11:05:32.229 INFO:teuthology.orchestra.run.smithi102.stdout: Package Arch Version Repo Size 2024-09-16T11:05:32.229 INFO:teuthology.orchestra.run.smithi102.stdout:================================================================================ 2024-09-16T11:05:32.230 INFO:teuthology.orchestra.run.smithi102.stdout:Installing: 2024-09-16T11:05:32.230 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-mon-debuginfo x86_64 2:17.2.7-1643.gd75f2382.el9 ceph 47 M 2024-09-16T11:05:32.230 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-16T11:05:32.230 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction Summary 2024-09-16T11:05:32.230 INFO:teuthology.orchestra.run.smithi102.stdout:================================================================================ 2024-09-16T11:05:32.230 INFO:teuthology.orchestra.run.smithi102.stdout:Install 1 Package 2024-09-16T11:05:32.230 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-16T11:05:32.230 INFO:teuthology.orchestra.run.smithi102.stdout:Total download size: 47 M 2024-09-16T11:05:32.230 INFO:teuthology.orchestra.run.smithi102.stdout:Installed size: 290 M 2024-09-16T11:05:32.230 INFO:teuthology.orchestra.run.smithi102.stdout:Downloading Packages: 2024-09-16T11:05:34.417 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-mon-debuginfo-17.2.7-1643.gd75f2382.el9.x8 21 MB/s | 47 MB 00:02 2024-09-16T11:05:34.418 INFO:teuthology.orchestra.run.smithi102.stdout:-------------------------------------------------------------------------------- 2024-09-16T11:05:34.418 INFO:teuthology.orchestra.run.smithi102.stdout:Total 21 MB/s | 47 MB 00:02 2024-09-16T11:05:34.418 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction check 2024-09-16T11:05:34.422 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction check succeeded. 2024-09-16T11:05:34.422 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction test 2024-09-16T11:05:34.732 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction test succeeded. 2024-09-16T11:05:34.732 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction 2024-09-16T11:05:40.227 INFO:teuthology.orchestra.run.smithi102.stdout: Preparing : 1/1 2024-09-16T11:05:40.322 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : ceph-mon-debuginfo-2:17.2.7-1643.gd75f2382.el9.x86_6 1/1 2024-09-16T11:05:42.406 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: ceph-mon-debuginfo-2:17.2.7-1643.gd75f2382.el9.x86_6 1/1 2024-09-16T11:05:42.635 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : ceph-mon-debuginfo-2:17.2.7-1643.gd75f2382.el9.x86_6 1/1 2024-09-16T11:05:42.635 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-16T11:05:42.636 INFO:teuthology.orchestra.run.smithi102.stdout:Installed: 2024-09-16T11:05:42.636 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-mon-debuginfo-2:17.2.7-1643.gd75f2382.el9.x86_64 2024-09-16T11:05:42.636 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-16T11:05:42.636 INFO:teuthology.orchestra.run.smithi102.stdout:Complete! 2024-09-16T11:05:42.823 DEBUG:teuthology.orchestra.run.smithi102:> sudo yum -y install ceph-osd-debuginfo 2024-09-16T11:05:43.393 INFO:teuthology.orchestra.run.smithi102.stdout:Last metadata expiration check: 0:06:33 ago on Mon 16 Sep 2024 10:59:10 AM UTC. 2024-09-16T11:05:43.527 INFO:teuthology.orchestra.run.smithi102.stdout:Dependencies resolved. 2024-09-16T11:05:43.527 INFO:teuthology.orchestra.run.smithi102.stdout:================================================================================ 2024-09-16T11:05:43.527 INFO:teuthology.orchestra.run.smithi102.stdout: Package Arch Version Repo Size 2024-09-16T11:05:43.528 INFO:teuthology.orchestra.run.smithi102.stdout:================================================================================ 2024-09-16T11:05:43.528 INFO:teuthology.orchestra.run.smithi102.stdout:Installing: 2024-09-16T11:05:43.528 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-osd-debuginfo x86_64 2:17.2.7-1643.gd75f2382.el9 ceph 192 M 2024-09-16T11:05:43.528 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-16T11:05:43.528 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction Summary 2024-09-16T11:05:43.528 INFO:teuthology.orchestra.run.smithi102.stdout:================================================================================ 2024-09-16T11:05:43.528 INFO:teuthology.orchestra.run.smithi102.stdout:Install 1 Package 2024-09-16T11:05:43.528 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-16T11:05:43.529 INFO:teuthology.orchestra.run.smithi102.stdout:Total download size: 192 M 2024-09-16T11:05:43.529 INFO:teuthology.orchestra.run.smithi102.stdout:Installed size: 901 M 2024-09-16T11:05:43.529 INFO:teuthology.orchestra.run.smithi102.stdout:Downloading Packages: 2024-09-16T11:05:52.302 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-osd-debuginfo-17.2.7-1643.gd75f2382.el9.x8 22 MB/s | 192 MB 00:08 2024-09-16T11:05:52.303 INFO:teuthology.orchestra.run.smithi102.stdout:-------------------------------------------------------------------------------- 2024-09-16T11:05:52.303 INFO:teuthology.orchestra.run.smithi102.stdout:Total 22 MB/s | 192 MB 00:08 2024-09-16T11:05:52.303 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction check 2024-09-16T11:05:52.307 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction check succeeded. 2024-09-16T11:05:52.307 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction test 2024-09-16T11:05:53.463 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction test succeeded. 2024-09-16T11:05:53.464 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction 2024-09-16T11:06:13.564 INFO:teuthology.orchestra.run.smithi102.stdout: Preparing : 1/1 2024-09-16T11:06:13.657 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : ceph-osd-debuginfo-2:17.2.7-1643.gd75f2382.el9.x86_6 1/1 2024-09-16T11:06:19.182 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: ceph-osd-debuginfo-2:17.2.7-1643.gd75f2382.el9.x86_6 1/1 2024-09-16T11:06:19.408 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : ceph-osd-debuginfo-2:17.2.7-1643.gd75f2382.el9.x86_6 1/1 2024-09-16T11:06:19.408 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-16T11:06:19.409 INFO:teuthology.orchestra.run.smithi102.stdout:Installed: 2024-09-16T11:06:19.409 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-osd-debuginfo-2:17.2.7-1643.gd75f2382.el9.x86_64 2024-09-16T11:06:19.409 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-16T11:06:19.409 INFO:teuthology.orchestra.run.smithi102.stdout:Complete! 2024-09-16T11:06:19.614 DEBUG:teuthology.orchestra.run.smithi102:> sudo yum -y install ceph-fuse-debuginfo 2024-09-16T11:06:20.185 INFO:teuthology.orchestra.run.smithi102.stdout:Last metadata expiration check: 0:07:10 ago on Mon 16 Sep 2024 10:59:10 AM UTC. 2024-09-16T11:06:20.320 INFO:teuthology.orchestra.run.smithi102.stdout:Dependencies resolved. 2024-09-16T11:06:20.321 INFO:teuthology.orchestra.run.smithi102.stdout:================================================================================ 2024-09-16T11:06:20.321 INFO:teuthology.orchestra.run.smithi102.stdout: Package Arch Version Repo Size 2024-09-16T11:06:20.321 INFO:teuthology.orchestra.run.smithi102.stdout:================================================================================ 2024-09-16T11:06:20.321 INFO:teuthology.orchestra.run.smithi102.stdout:Installing: 2024-09-16T11:06:20.321 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-fuse-debuginfo x86_64 2:17.2.7-1643.gd75f2382.el9 ceph 5.6 M 2024-09-16T11:06:20.321 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-16T11:06:20.321 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction Summary 2024-09-16T11:06:20.321 INFO:teuthology.orchestra.run.smithi102.stdout:================================================================================ 2024-09-16T11:06:20.321 INFO:teuthology.orchestra.run.smithi102.stdout:Install 1 Package 2024-09-16T11:06:20.322 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-16T11:06:20.322 INFO:teuthology.orchestra.run.smithi102.stdout:Total download size: 5.6 M 2024-09-16T11:06:20.322 INFO:teuthology.orchestra.run.smithi102.stdout:Installed size: 27 M 2024-09-16T11:06:20.322 INFO:teuthology.orchestra.run.smithi102.stdout:Downloading Packages: 2024-09-16T11:06:20.787 INFO:teuthology.orchestra.run.smithi102.stdout:ceph-fuse-debuginfo-17.2.7-1643.gd75f2382.el9.x 12 MB/s | 5.6 MB 00:00 2024-09-16T11:06:20.787 INFO:teuthology.orchestra.run.smithi102.stdout:-------------------------------------------------------------------------------- 2024-09-16T11:06:20.788 INFO:teuthology.orchestra.run.smithi102.stdout:Total 12 MB/s | 5.6 MB 00:00 2024-09-16T11:06:20.788 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction check 2024-09-16T11:06:20.791 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction check succeeded. 2024-09-16T11:06:20.791 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction test 2024-09-16T11:06:20.860 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction test succeeded. 2024-09-16T11:06:20.860 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction 2024-09-16T11:06:21.557 INFO:teuthology.orchestra.run.smithi102.stdout: Preparing : 1/1 2024-09-16T11:06:21.659 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : ceph-fuse-debuginfo-2:17.2.7-1643.gd75f2382.el9.x86_ 1/1 2024-09-16T11:06:22.154 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: ceph-fuse-debuginfo-2:17.2.7-1643.gd75f2382.el9.x86_ 1/1 2024-09-16T11:06:22.399 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : ceph-fuse-debuginfo-2:17.2.7-1643.gd75f2382.el9.x86_ 1/1 2024-09-16T11:06:22.399 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-16T11:06:22.399 INFO:teuthology.orchestra.run.smithi102.stdout:Installed: 2024-09-16T11:06:22.399 INFO:teuthology.orchestra.run.smithi102.stdout: ceph-fuse-debuginfo-2:17.2.7-1643.gd75f2382.el9.x86_64 2024-09-16T11:06:22.399 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-16T11:06:22.400 INFO:teuthology.orchestra.run.smithi102.stdout:Complete! 2024-09-16T11:06:22.577 DEBUG:teuthology.orchestra.run.smithi102:> sudo yum -y install librados-devel-debuginfo 2024-09-16T11:06:23.145 INFO:teuthology.orchestra.run.smithi102.stdout:Last metadata expiration check: 0:07:13 ago on Mon 16 Sep 2024 10:59:10 AM UTC. 2024-09-16T11:06:23.277 INFO:teuthology.orchestra.run.smithi102.stdout:Dependencies resolved. 2024-09-16T11:06:23.278 INFO:teuthology.orchestra.run.smithi102.stdout:================================================================================ 2024-09-16T11:06:23.278 INFO:teuthology.orchestra.run.smithi102.stdout: Package Arch Version Repo Size 2024-09-16T11:06:23.279 INFO:teuthology.orchestra.run.smithi102.stdout:================================================================================ 2024-09-16T11:06:23.279 INFO:teuthology.orchestra.run.smithi102.stdout:Installing: 2024-09-16T11:06:23.279 INFO:teuthology.orchestra.run.smithi102.stdout: librados-devel-debuginfo x86_64 2:17.2.7-1643.gd75f2382.el9 ceph 106 k 2024-09-16T11:06:23.279 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-16T11:06:23.279 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction Summary 2024-09-16T11:06:23.279 INFO:teuthology.orchestra.run.smithi102.stdout:================================================================================ 2024-09-16T11:06:23.279 INFO:teuthology.orchestra.run.smithi102.stdout:Install 1 Package 2024-09-16T11:06:23.279 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-16T11:06:23.280 INFO:teuthology.orchestra.run.smithi102.stdout:Total download size: 106 k 2024-09-16T11:06:23.280 INFO:teuthology.orchestra.run.smithi102.stdout:Installed size: 437 k 2024-09-16T11:06:23.280 INFO:teuthology.orchestra.run.smithi102.stdout:Downloading Packages: 2024-09-16T11:06:23.473 INFO:teuthology.orchestra.run.smithi102.stdout:librados-devel-debuginfo-17.2.7-1643.gd75f2382. 548 kB/s | 106 kB 00:00 2024-09-16T11:06:23.473 INFO:teuthology.orchestra.run.smithi102.stdout:-------------------------------------------------------------------------------- 2024-09-16T11:06:23.474 INFO:teuthology.orchestra.run.smithi102.stdout:Total 544 kB/s | 106 kB 00:00 2024-09-16T11:06:23.474 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction check 2024-09-16T11:06:23.477 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction check succeeded. 2024-09-16T11:06:23.477 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction test 2024-09-16T11:06:23.512 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction test succeeded. 2024-09-16T11:06:23.513 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction 2024-09-16T11:06:23.631 INFO:teuthology.orchestra.run.smithi102.stdout: Preparing : 1/1 2024-09-16T11:06:23.753 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : librados-devel-debuginfo-2:17.2.7-1643.gd75f2382.el9 1/1 2024-09-16T11:06:24.095 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: librados-devel-debuginfo-2:17.2.7-1643.gd75f2382.el9 1/1 2024-09-16T11:06:24.330 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : librados-devel-debuginfo-2:17.2.7-1643.gd75f2382.el9 1/1 2024-09-16T11:06:24.330 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-16T11:06:24.330 INFO:teuthology.orchestra.run.smithi102.stdout:Installed: 2024-09-16T11:06:24.331 INFO:teuthology.orchestra.run.smithi102.stdout: librados-devel-debuginfo-2:17.2.7-1643.gd75f2382.el9.x86_64 2024-09-16T11:06:24.331 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-16T11:06:24.331 INFO:teuthology.orchestra.run.smithi102.stdout:Complete! 2024-09-16T11:06:24.495 DEBUG:teuthology.orchestra.run.smithi102:> sudo yum -y install libcephfs2-debuginfo 2024-09-16T11:06:25.057 INFO:teuthology.orchestra.run.smithi102.stdout:Last metadata expiration check: 0:07:15 ago on Mon 16 Sep 2024 10:59:10 AM UTC. 2024-09-16T11:06:25.191 INFO:teuthology.orchestra.run.smithi102.stdout:Dependencies resolved. 2024-09-16T11:06:25.192 INFO:teuthology.orchestra.run.smithi102.stdout:================================================================================ 2024-09-16T11:06:25.192 INFO:teuthology.orchestra.run.smithi102.stdout: Package Arch Version Repo Size 2024-09-16T11:06:25.193 INFO:teuthology.orchestra.run.smithi102.stdout:================================================================================ 2024-09-16T11:06:25.193 INFO:teuthology.orchestra.run.smithi102.stdout:Installing: 2024-09-16T11:06:25.193 INFO:teuthology.orchestra.run.smithi102.stdout: libcephfs2-debuginfo x86_64 2:17.2.7-1643.gd75f2382.el9 ceph 4.8 M 2024-09-16T11:06:25.193 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-16T11:06:25.193 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction Summary 2024-09-16T11:06:25.193 INFO:teuthology.orchestra.run.smithi102.stdout:================================================================================ 2024-09-16T11:06:25.193 INFO:teuthology.orchestra.run.smithi102.stdout:Install 1 Package 2024-09-16T11:06:25.193 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-16T11:06:25.193 INFO:teuthology.orchestra.run.smithi102.stdout:Total download size: 4.8 M 2024-09-16T11:06:25.193 INFO:teuthology.orchestra.run.smithi102.stdout:Installed size: 23 M 2024-09-16T11:06:25.193 INFO:teuthology.orchestra.run.smithi102.stdout:Downloading Packages: 2024-09-16T11:06:25.626 INFO:teuthology.orchestra.run.smithi102.stdout:libcephfs2-debuginfo-17.2.7-1643.gd75f2382.el9. 11 MB/s | 4.8 MB 00:00 2024-09-16T11:06:25.627 INFO:teuthology.orchestra.run.smithi102.stdout:-------------------------------------------------------------------------------- 2024-09-16T11:06:25.627 INFO:teuthology.orchestra.run.smithi102.stdout:Total 11 MB/s | 4.8 MB 00:00 2024-09-16T11:06:25.628 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction check 2024-09-16T11:06:25.631 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction check succeeded. 2024-09-16T11:06:25.631 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction test 2024-09-16T11:06:25.695 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction test succeeded. 2024-09-16T11:06:25.695 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction 2024-09-16T11:06:26.308 INFO:teuthology.orchestra.run.smithi102.stdout: Preparing : 1/1 2024-09-16T11:06:26.406 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : libcephfs2-debuginfo-2:17.2.7-1643.gd75f2382.el9.x86 1/1 2024-09-16T11:06:26.902 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: libcephfs2-debuginfo-2:17.2.7-1643.gd75f2382.el9.x86 1/1 2024-09-16T11:06:27.138 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : libcephfs2-debuginfo-2:17.2.7-1643.gd75f2382.el9.x86 1/1 2024-09-16T11:06:27.139 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-16T11:06:27.139 INFO:teuthology.orchestra.run.smithi102.stdout:Installed: 2024-09-16T11:06:27.139 INFO:teuthology.orchestra.run.smithi102.stdout: libcephfs2-debuginfo-2:17.2.7-1643.gd75f2382.el9.x86_64 2024-09-16T11:06:27.139 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-16T11:06:27.139 INFO:teuthology.orchestra.run.smithi102.stdout:Complete! 2024-09-16T11:06:27.315 DEBUG:teuthology.orchestra.run.smithi102:> sudo yum -y install librados2-debuginfo 2024-09-16T11:06:27.883 INFO:teuthology.orchestra.run.smithi102.stdout:Last metadata expiration check: 0:07:17 ago on Mon 16 Sep 2024 10:59:10 AM UTC. 2024-09-16T11:06:28.018 INFO:teuthology.orchestra.run.smithi102.stdout:Dependencies resolved. 2024-09-16T11:06:28.019 INFO:teuthology.orchestra.run.smithi102.stdout:================================================================================ 2024-09-16T11:06:28.019 INFO:teuthology.orchestra.run.smithi102.stdout: Package Arch Version Repo Size 2024-09-16T11:06:28.019 INFO:teuthology.orchestra.run.smithi102.stdout:================================================================================ 2024-09-16T11:06:28.019 INFO:teuthology.orchestra.run.smithi102.stdout:Installing: 2024-09-16T11:06:28.019 INFO:teuthology.orchestra.run.smithi102.stdout: librados2-debuginfo x86_64 2:17.2.7-1643.gd75f2382.el9 ceph 42 M 2024-09-16T11:06:28.019 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-16T11:06:28.019 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction Summary 2024-09-16T11:06:28.019 INFO:teuthology.orchestra.run.smithi102.stdout:================================================================================ 2024-09-16T11:06:28.019 INFO:teuthology.orchestra.run.smithi102.stdout:Install 1 Package 2024-09-16T11:06:28.019 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-16T11:06:28.019 INFO:teuthology.orchestra.run.smithi102.stdout:Total download size: 42 M 2024-09-16T11:06:28.020 INFO:teuthology.orchestra.run.smithi102.stdout:Installed size: 234 M 2024-09-16T11:06:28.020 INFO:teuthology.orchestra.run.smithi102.stdout:Downloading Packages: 2024-09-16T11:06:29.896 INFO:teuthology.orchestra.run.smithi102.stdout:librados2-debuginfo-17.2.7-1643.gd75f2382.el9.x 23 MB/s | 42 MB 00:01 2024-09-16T11:06:29.896 INFO:teuthology.orchestra.run.smithi102.stdout:-------------------------------------------------------------------------------- 2024-09-16T11:06:29.897 INFO:teuthology.orchestra.run.smithi102.stdout:Total 23 MB/s | 42 MB 00:01 2024-09-16T11:06:29.897 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction check 2024-09-16T11:06:29.901 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction check succeeded. 2024-09-16T11:06:29.901 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction test 2024-09-16T11:06:30.192 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction test succeeded. 2024-09-16T11:06:30.192 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction 2024-09-16T11:06:34.985 INFO:teuthology.orchestra.run.smithi102.stdout: Preparing : 1/1 2024-09-16T11:06:35.069 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : librados2-debuginfo-2:17.2.7-1643.gd75f2382.el9.x86_ 1/1 2024-09-16T11:06:36.788 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: librados2-debuginfo-2:17.2.7-1643.gd75f2382.el9.x86_ 1/1 2024-09-16T11:06:37.041 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : librados2-debuginfo-2:17.2.7-1643.gd75f2382.el9.x86_ 1/1 2024-09-16T11:06:37.041 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-16T11:06:37.041 INFO:teuthology.orchestra.run.smithi102.stdout:Installed: 2024-09-16T11:06:37.042 INFO:teuthology.orchestra.run.smithi102.stdout: librados2-debuginfo-2:17.2.7-1643.gd75f2382.el9.x86_64 2024-09-16T11:06:37.042 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-16T11:06:37.042 INFO:teuthology.orchestra.run.smithi102.stdout:Complete! 2024-09-16T11:06:37.228 DEBUG:teuthology.orchestra.run.smithi102:> sudo yum -y install librbd1-debuginfo 2024-09-16T11:06:37.791 INFO:teuthology.orchestra.run.smithi102.stdout:Last metadata expiration check: 0:07:27 ago on Mon 16 Sep 2024 10:59:10 AM UTC. 2024-09-16T11:06:37.923 INFO:teuthology.orchestra.run.smithi102.stdout:Dependencies resolved. 2024-09-16T11:06:37.924 INFO:teuthology.orchestra.run.smithi102.stdout:================================================================================ 2024-09-16T11:06:37.924 INFO:teuthology.orchestra.run.smithi102.stdout: Package Arch Version Repo Size 2024-09-16T11:06:37.924 INFO:teuthology.orchestra.run.smithi102.stdout:================================================================================ 2024-09-16T11:06:37.924 INFO:teuthology.orchestra.run.smithi102.stdout:Installing: 2024-09-16T11:06:37.924 INFO:teuthology.orchestra.run.smithi102.stdout: librbd1-debuginfo x86_64 2:17.2.7-1643.gd75f2382.el9 ceph 50 M 2024-09-16T11:06:37.924 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-16T11:06:37.924 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction Summary 2024-09-16T11:06:37.924 INFO:teuthology.orchestra.run.smithi102.stdout:================================================================================ 2024-09-16T11:06:37.924 INFO:teuthology.orchestra.run.smithi102.stdout:Install 1 Package 2024-09-16T11:06:37.925 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-16T11:06:37.925 INFO:teuthology.orchestra.run.smithi102.stdout:Total download size: 50 M 2024-09-16T11:06:37.925 INFO:teuthology.orchestra.run.smithi102.stdout:Installed size: 285 M 2024-09-16T11:06:37.925 INFO:teuthology.orchestra.run.smithi102.stdout:Downloading Packages: 2024-09-16T11:06:40.149 INFO:teuthology.orchestra.run.smithi102.stdout:librbd1-debuginfo-17.2.7-1643.gd75f2382.el9.x86 23 MB/s | 50 MB 00:02 2024-09-16T11:06:40.150 INFO:teuthology.orchestra.run.smithi102.stdout:-------------------------------------------------------------------------------- 2024-09-16T11:06:40.150 INFO:teuthology.orchestra.run.smithi102.stdout:Total 23 MB/s | 50 MB 00:02 2024-09-16T11:06:40.150 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction check 2024-09-16T11:06:40.154 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction check succeeded. 2024-09-16T11:06:40.154 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction test 2024-09-16T11:06:40.489 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction test succeeded. 2024-09-16T11:06:40.489 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction 2024-09-16T11:06:46.203 INFO:teuthology.orchestra.run.smithi102.stdout: Preparing : 1/1 2024-09-16T11:06:46.296 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : librbd1-debuginfo-2:17.2.7-1643.gd75f2382.el9.x86_64 1/1 2024-09-16T11:06:48.335 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: librbd1-debuginfo-2:17.2.7-1643.gd75f2382.el9.x86_64 1/1 2024-09-16T11:06:48.567 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : librbd1-debuginfo-2:17.2.7-1643.gd75f2382.el9.x86_64 1/1 2024-09-16T11:06:48.567 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-16T11:06:48.567 INFO:teuthology.orchestra.run.smithi102.stdout:Installed: 2024-09-16T11:06:48.567 INFO:teuthology.orchestra.run.smithi102.stdout: librbd1-debuginfo-2:17.2.7-1643.gd75f2382.el9.x86_64 2024-09-16T11:06:48.567 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-16T11:06:48.567 INFO:teuthology.orchestra.run.smithi102.stdout:Complete! 2024-09-16T11:06:48.735 DEBUG:teuthology.orchestra.run.smithi102:> sudo yum -y install python3-cephfs-debuginfo 2024-09-16T11:06:49.299 INFO:teuthology.orchestra.run.smithi102.stdout:Last metadata expiration check: 0:07:39 ago on Mon 16 Sep 2024 10:59:10 AM UTC. 2024-09-16T11:06:49.431 INFO:teuthology.orchestra.run.smithi102.stdout:Dependencies resolved. 2024-09-16T11:06:49.432 INFO:teuthology.orchestra.run.smithi102.stdout:================================================================================ 2024-09-16T11:06:49.432 INFO:teuthology.orchestra.run.smithi102.stdout: Package Arch Version Repo Size 2024-09-16T11:06:49.432 INFO:teuthology.orchestra.run.smithi102.stdout:================================================================================ 2024-09-16T11:06:49.432 INFO:teuthology.orchestra.run.smithi102.stdout:Installing: 2024-09-16T11:06:49.432 INFO:teuthology.orchestra.run.smithi102.stdout: python3-cephfs-debuginfo x86_64 2:17.2.7-1643.gd75f2382.el9 ceph 593 k 2024-09-16T11:06:49.432 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-16T11:06:49.432 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction Summary 2024-09-16T11:06:49.432 INFO:teuthology.orchestra.run.smithi102.stdout:================================================================================ 2024-09-16T11:06:49.432 INFO:teuthology.orchestra.run.smithi102.stdout:Install 1 Package 2024-09-16T11:06:49.433 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-16T11:06:49.433 INFO:teuthology.orchestra.run.smithi102.stdout:Total download size: 593 k 2024-09-16T11:06:49.433 INFO:teuthology.orchestra.run.smithi102.stdout:Installed size: 2.0 M 2024-09-16T11:06:49.433 INFO:teuthology.orchestra.run.smithi102.stdout:Downloading Packages: 2024-09-16T11:06:49.694 INFO:teuthology.orchestra.run.smithi102.stdout:python3-cephfs-debuginfo-17.2.7-1643.gd75f2382. 2.2 MB/s | 593 kB 00:00 2024-09-16T11:06:49.694 INFO:teuthology.orchestra.run.smithi102.stdout:-------------------------------------------------------------------------------- 2024-09-16T11:06:49.695 INFO:teuthology.orchestra.run.smithi102.stdout:Total 2.2 MB/s | 593 kB 00:00 2024-09-16T11:06:49.695 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction check 2024-09-16T11:06:49.698 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction check succeeded. 2024-09-16T11:06:49.698 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction test 2024-09-16T11:06:49.736 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction test succeeded. 2024-09-16T11:06:49.736 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction 2024-09-16T11:06:49.902 INFO:teuthology.orchestra.run.smithi102.stdout: Preparing : 1/1 2024-09-16T11:06:49.991 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-cephfs-debuginfo-2:17.2.7-1643.gd75f2382.el9 1/1 2024-09-16T11:06:50.381 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: python3-cephfs-debuginfo-2:17.2.7-1643.gd75f2382.el9 1/1 2024-09-16T11:06:50.618 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-cephfs-debuginfo-2:17.2.7-1643.gd75f2382.el9 1/1 2024-09-16T11:06:50.618 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-16T11:06:50.618 INFO:teuthology.orchestra.run.smithi102.stdout:Installed: 2024-09-16T11:06:50.618 INFO:teuthology.orchestra.run.smithi102.stdout: python3-cephfs-debuginfo-2:17.2.7-1643.gd75f2382.el9.x86_64 2024-09-16T11:06:50.618 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-16T11:06:50.618 INFO:teuthology.orchestra.run.smithi102.stdout:Complete! 2024-09-16T11:06:50.793 DEBUG:teuthology.orchestra.run.smithi102:> sudo yum -y install python3-rados-debuginfo 2024-09-16T11:06:51.359 INFO:teuthology.orchestra.run.smithi102.stdout:Last metadata expiration check: 0:07:41 ago on Mon 16 Sep 2024 10:59:10 AM UTC. 2024-09-16T11:06:51.492 INFO:teuthology.orchestra.run.smithi102.stdout:Dependencies resolved. 2024-09-16T11:06:51.493 INFO:teuthology.orchestra.run.smithi102.stdout:================================================================================ 2024-09-16T11:06:51.493 INFO:teuthology.orchestra.run.smithi102.stdout: Package Arch Version Repo Size 2024-09-16T11:06:51.493 INFO:teuthology.orchestra.run.smithi102.stdout:================================================================================ 2024-09-16T11:06:51.493 INFO:teuthology.orchestra.run.smithi102.stdout:Installing: 2024-09-16T11:06:51.493 INFO:teuthology.orchestra.run.smithi102.stdout: python3-rados-debuginfo x86_64 2:17.2.7-1643.gd75f2382.el9 ceph 768 k 2024-09-16T11:06:51.494 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-16T11:06:51.494 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction Summary 2024-09-16T11:06:51.494 INFO:teuthology.orchestra.run.smithi102.stdout:================================================================================ 2024-09-16T11:06:51.494 INFO:teuthology.orchestra.run.smithi102.stdout:Install 1 Package 2024-09-16T11:06:51.494 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-16T11:06:51.494 INFO:teuthology.orchestra.run.smithi102.stdout:Total download size: 768 k 2024-09-16T11:06:51.494 INFO:teuthology.orchestra.run.smithi102.stdout:Installed size: 2.7 M 2024-09-16T11:06:51.494 INFO:teuthology.orchestra.run.smithi102.stdout:Downloading Packages: 2024-09-16T11:06:51.897 INFO:teuthology.orchestra.run.smithi102.stdout:python3-rados-debuginfo-17.2.7-1643.gd75f2382.e 1.9 MB/s | 768 kB 00:00 2024-09-16T11:06:51.898 INFO:teuthology.orchestra.run.smithi102.stdout:-------------------------------------------------------------------------------- 2024-09-16T11:06:51.898 INFO:teuthology.orchestra.run.smithi102.stdout:Total 1.9 MB/s | 768 kB 00:00 2024-09-16T11:06:51.898 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction check 2024-09-16T11:06:51.902 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction check succeeded. 2024-09-16T11:06:51.902 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction test 2024-09-16T11:06:51.942 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction test succeeded. 2024-09-16T11:06:51.942 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction 2024-09-16T11:06:52.185 INFO:teuthology.orchestra.run.smithi102.stdout: Preparing : 1/1 2024-09-16T11:06:52.286 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-rados-debuginfo-2:17.2.7-1643.gd75f2382.el9. 1/1 2024-09-16T11:06:52.652 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: python3-rados-debuginfo-2:17.2.7-1643.gd75f2382.el9. 1/1 2024-09-16T11:06:52.883 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-rados-debuginfo-2:17.2.7-1643.gd75f2382.el9. 1/1 2024-09-16T11:06:52.884 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-16T11:06:52.884 INFO:teuthology.orchestra.run.smithi102.stdout:Installed: 2024-09-16T11:06:52.884 INFO:teuthology.orchestra.run.smithi102.stdout: python3-rados-debuginfo-2:17.2.7-1643.gd75f2382.el9.x86_64 2024-09-16T11:06:52.884 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-16T11:06:52.884 INFO:teuthology.orchestra.run.smithi102.stdout:Complete! 2024-09-16T11:06:53.047 DEBUG:teuthology.orchestra.run.smithi102:> sudo yum -y install python3-rbd-debuginfo 2024-09-16T11:06:53.612 INFO:teuthology.orchestra.run.smithi102.stdout:Last metadata expiration check: 0:07:43 ago on Mon 16 Sep 2024 10:59:10 AM UTC. 2024-09-16T11:06:53.743 INFO:teuthology.orchestra.run.smithi102.stdout:Dependencies resolved. 2024-09-16T11:06:53.744 INFO:teuthology.orchestra.run.smithi102.stdout:================================================================================ 2024-09-16T11:06:53.744 INFO:teuthology.orchestra.run.smithi102.stdout: Package Arch Version Repo Size 2024-09-16T11:06:53.744 INFO:teuthology.orchestra.run.smithi102.stdout:================================================================================ 2024-09-16T11:06:53.745 INFO:teuthology.orchestra.run.smithi102.stdout:Installing: 2024-09-16T11:06:53.745 INFO:teuthology.orchestra.run.smithi102.stdout: python3-rbd-debuginfo x86_64 2:17.2.7-1643.gd75f2382.el9 ceph 1.1 M 2024-09-16T11:06:53.745 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-16T11:06:53.745 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction Summary 2024-09-16T11:06:53.745 INFO:teuthology.orchestra.run.smithi102.stdout:================================================================================ 2024-09-16T11:06:53.745 INFO:teuthology.orchestra.run.smithi102.stdout:Install 1 Package 2024-09-16T11:06:53.745 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-16T11:06:53.745 INFO:teuthology.orchestra.run.smithi102.stdout:Total download size: 1.1 M 2024-09-16T11:06:53.745 INFO:teuthology.orchestra.run.smithi102.stdout:Installed size: 4.2 M 2024-09-16T11:06:53.745 INFO:teuthology.orchestra.run.smithi102.stdout:Downloading Packages: 2024-09-16T11:06:54.058 INFO:teuthology.orchestra.run.smithi102.stdout:python3-rbd-debuginfo-17.2.7-1643.gd75f2382.el9 3.5 MB/s | 1.1 MB 00:00 2024-09-16T11:06:54.059 INFO:teuthology.orchestra.run.smithi102.stdout:-------------------------------------------------------------------------------- 2024-09-16T11:06:54.059 INFO:teuthology.orchestra.run.smithi102.stdout:Total 3.5 MB/s | 1.1 MB 00:00 2024-09-16T11:06:54.060 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction check 2024-09-16T11:06:54.063 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction check succeeded. 2024-09-16T11:06:54.063 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction test 2024-09-16T11:06:54.103 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction test succeeded. 2024-09-16T11:06:54.104 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction 2024-09-16T11:06:54.323 INFO:teuthology.orchestra.run.smithi102.stdout: Preparing : 1/1 2024-09-16T11:06:54.414 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-rbd-debuginfo-2:17.2.7-1643.gd75f2382.el9.x8 1/1 2024-09-16T11:06:54.777 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: python3-rbd-debuginfo-2:17.2.7-1643.gd75f2382.el9.x8 1/1 2024-09-16T11:06:55.009 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-rbd-debuginfo-2:17.2.7-1643.gd75f2382.el9.x8 1/1 2024-09-16T11:06:55.010 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-16T11:06:55.010 INFO:teuthology.orchestra.run.smithi102.stdout:Installed: 2024-09-16T11:06:55.010 INFO:teuthology.orchestra.run.smithi102.stdout: python3-rbd-debuginfo-2:17.2.7-1643.gd75f2382.el9.x86_64 2024-09-16T11:06:55.010 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-16T11:06:55.010 INFO:teuthology.orchestra.run.smithi102.stdout:Complete! 2024-09-16T11:06:55.191 DEBUG:teuthology.orchestra.run.smithi102:> sudo yum -y install python3-rgw-debuginfo 2024-09-16T11:06:55.759 INFO:teuthology.orchestra.run.smithi102.stdout:Last metadata expiration check: 0:07:45 ago on Mon 16 Sep 2024 10:59:10 AM UTC. 2024-09-16T11:06:55.891 INFO:teuthology.orchestra.run.smithi102.stdout:Dependencies resolved. 2024-09-16T11:06:55.892 INFO:teuthology.orchestra.run.smithi102.stdout:================================================================================ 2024-09-16T11:06:55.892 INFO:teuthology.orchestra.run.smithi102.stdout: Package Arch Version Repo Size 2024-09-16T11:06:55.892 INFO:teuthology.orchestra.run.smithi102.stdout:================================================================================ 2024-09-16T11:06:55.892 INFO:teuthology.orchestra.run.smithi102.stdout:Installing: 2024-09-16T11:06:55.892 INFO:teuthology.orchestra.run.smithi102.stdout: python3-rgw-debuginfo x86_64 2:17.2.7-1643.gd75f2382.el9 ceph 337 k 2024-09-16T11:06:55.892 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-16T11:06:55.892 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction Summary 2024-09-16T11:06:55.892 INFO:teuthology.orchestra.run.smithi102.stdout:================================================================================ 2024-09-16T11:06:55.892 INFO:teuthology.orchestra.run.smithi102.stdout:Install 1 Package 2024-09-16T11:06:55.892 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-16T11:06:55.893 INFO:teuthology.orchestra.run.smithi102.stdout:Total download size: 337 k 2024-09-16T11:06:55.893 INFO:teuthology.orchestra.run.smithi102.stdout:Installed size: 1.0 M 2024-09-16T11:06:55.893 INFO:teuthology.orchestra.run.smithi102.stdout:Downloading Packages: 2024-09-16T11:06:56.153 INFO:teuthology.orchestra.run.smithi102.stdout:python3-rgw-debuginfo-17.2.7-1643.gd75f2382.el9 1.3 MB/s | 337 kB 00:00 2024-09-16T11:06:56.154 INFO:teuthology.orchestra.run.smithi102.stdout:-------------------------------------------------------------------------------- 2024-09-16T11:06:56.154 INFO:teuthology.orchestra.run.smithi102.stdout:Total 1.3 MB/s | 337 kB 00:00 2024-09-16T11:06:56.154 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction check 2024-09-16T11:06:56.158 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction check succeeded. 2024-09-16T11:06:56.158 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction test 2024-09-16T11:06:56.193 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction test succeeded. 2024-09-16T11:06:56.194 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction 2024-09-16T11:06:56.327 INFO:teuthology.orchestra.run.smithi102.stdout: Preparing : 1/1 2024-09-16T11:06:56.433 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : python3-rgw-debuginfo-2:17.2.7-1643.gd75f2382.el9.x8 1/1 2024-09-16T11:06:56.797 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: python3-rgw-debuginfo-2:17.2.7-1643.gd75f2382.el9.x8 1/1 2024-09-16T11:06:57.032 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : python3-rgw-debuginfo-2:17.2.7-1643.gd75f2382.el9.x8 1/1 2024-09-16T11:06:57.033 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-16T11:06:57.033 INFO:teuthology.orchestra.run.smithi102.stdout:Installed: 2024-09-16T11:06:57.033 INFO:teuthology.orchestra.run.smithi102.stdout: python3-rgw-debuginfo-2:17.2.7-1643.gd75f2382.el9.x86_64 2024-09-16T11:06:57.033 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-16T11:06:57.033 INFO:teuthology.orchestra.run.smithi102.stdout:Complete! 2024-09-16T11:06:57.225 DEBUG:teuthology.orchestra.run.smithi102:> sudo yum -y install rbd-fuse-debuginfo 2024-09-16T11:06:57.789 INFO:teuthology.orchestra.run.smithi102.stdout:Last metadata expiration check: 0:07:47 ago on Mon 16 Sep 2024 10:59:10 AM UTC. 2024-09-16T11:06:57.920 INFO:teuthology.orchestra.run.smithi102.stdout:Dependencies resolved. 2024-09-16T11:06:57.922 INFO:teuthology.orchestra.run.smithi102.stdout:================================================================================ 2024-09-16T11:06:57.922 INFO:teuthology.orchestra.run.smithi102.stdout: Package Arch Version Repo Size 2024-09-16T11:06:57.922 INFO:teuthology.orchestra.run.smithi102.stdout:================================================================================ 2024-09-16T11:06:57.922 INFO:teuthology.orchestra.run.smithi102.stdout:Installing: 2024-09-16T11:06:57.922 INFO:teuthology.orchestra.run.smithi102.stdout: rbd-fuse-debuginfo x86_64 2:17.2.7-1643.gd75f2382.el9 ceph 496 k 2024-09-16T11:06:57.922 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-16T11:06:57.923 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction Summary 2024-09-16T11:06:57.923 INFO:teuthology.orchestra.run.smithi102.stdout:================================================================================ 2024-09-16T11:06:57.923 INFO:teuthology.orchestra.run.smithi102.stdout:Install 1 Package 2024-09-16T11:06:57.923 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-16T11:06:57.923 INFO:teuthology.orchestra.run.smithi102.stdout:Total download size: 496 k 2024-09-16T11:06:57.923 INFO:teuthology.orchestra.run.smithi102.stdout:Installed size: 2.5 M 2024-09-16T11:06:57.923 INFO:teuthology.orchestra.run.smithi102.stdout:Downloading Packages: 2024-09-16T11:06:58.190 INFO:teuthology.orchestra.run.smithi102.stdout:rbd-fuse-debuginfo-17.2.7-1643.gd75f2382.el9.x8 1.8 MB/s | 496 kB 00:00 2024-09-16T11:06:58.190 INFO:teuthology.orchestra.run.smithi102.stdout:-------------------------------------------------------------------------------- 2024-09-16T11:06:58.191 INFO:teuthology.orchestra.run.smithi102.stdout:Total 1.8 MB/s | 496 kB 00:00 2024-09-16T11:06:58.191 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction check 2024-09-16T11:06:58.194 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction check succeeded. 2024-09-16T11:06:58.194 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction test 2024-09-16T11:06:58.231 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction test succeeded. 2024-09-16T11:06:58.231 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction 2024-09-16T11:06:58.391 INFO:teuthology.orchestra.run.smithi102.stdout: Preparing : 1/1 2024-09-16T11:06:58.495 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : rbd-fuse-debuginfo-2:17.2.7-1643.gd75f2382.el9.x86_6 1/1 2024-09-16T11:06:58.867 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: rbd-fuse-debuginfo-2:17.2.7-1643.gd75f2382.el9.x86_6 1/1 2024-09-16T11:06:59.108 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : rbd-fuse-debuginfo-2:17.2.7-1643.gd75f2382.el9.x86_6 1/1 2024-09-16T11:06:59.108 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-16T11:06:59.108 INFO:teuthology.orchestra.run.smithi102.stdout:Installed: 2024-09-16T11:06:59.108 INFO:teuthology.orchestra.run.smithi102.stdout: rbd-fuse-debuginfo-2:17.2.7-1643.gd75f2382.el9.x86_64 2024-09-16T11:06:59.109 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-16T11:06:59.109 INFO:teuthology.orchestra.run.smithi102.stdout:Complete! 2024-09-16T11:06:59.313 DEBUG:teuthology.orchestra.run.smithi102:> sudo yum -y install rbd-mirror-debuginfo 2024-09-16T11:06:59.879 INFO:teuthology.orchestra.run.smithi102.stdout:Last metadata expiration check: 0:07:49 ago on Mon 16 Sep 2024 10:59:10 AM UTC. 2024-09-16T11:07:00.012 INFO:teuthology.orchestra.run.smithi102.stdout:Dependencies resolved. 2024-09-16T11:07:00.013 INFO:teuthology.orchestra.run.smithi102.stdout:================================================================================ 2024-09-16T11:07:00.013 INFO:teuthology.orchestra.run.smithi102.stdout: Package Arch Version Repo Size 2024-09-16T11:07:00.013 INFO:teuthology.orchestra.run.smithi102.stdout:================================================================================ 2024-09-16T11:07:00.013 INFO:teuthology.orchestra.run.smithi102.stdout:Installing: 2024-09-16T11:07:00.013 INFO:teuthology.orchestra.run.smithi102.stdout: rbd-mirror-debuginfo x86_64 2:17.2.7-1643.gd75f2382.el9 ceph 50 M 2024-09-16T11:07:00.013 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-16T11:07:00.013 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction Summary 2024-09-16T11:07:00.014 INFO:teuthology.orchestra.run.smithi102.stdout:================================================================================ 2024-09-16T11:07:00.014 INFO:teuthology.orchestra.run.smithi102.stdout:Install 1 Package 2024-09-16T11:07:00.014 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-16T11:07:00.014 INFO:teuthology.orchestra.run.smithi102.stdout:Total download size: 50 M 2024-09-16T11:07:00.014 INFO:teuthology.orchestra.run.smithi102.stdout:Installed size: 275 M 2024-09-16T11:07:00.014 INFO:teuthology.orchestra.run.smithi102.stdout:Downloading Packages: 2024-09-16T11:07:02.251 INFO:teuthology.orchestra.run.smithi102.stdout:rbd-mirror-debuginfo-17.2.7-1643.gd75f2382.el9. 22 MB/s | 50 MB 00:02 2024-09-16T11:07:02.252 INFO:teuthology.orchestra.run.smithi102.stdout:-------------------------------------------------------------------------------- 2024-09-16T11:07:02.252 INFO:teuthology.orchestra.run.smithi102.stdout:Total 22 MB/s | 50 MB 00:02 2024-09-16T11:07:02.253 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction check 2024-09-16T11:07:02.257 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction check succeeded. 2024-09-16T11:07:02.257 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction test 2024-09-16T11:07:02.585 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction test succeeded. 2024-09-16T11:07:02.585 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction 2024-09-16T11:07:08.205 INFO:teuthology.orchestra.run.smithi102.stdout: Preparing : 1/1 2024-09-16T11:07:08.281 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : rbd-mirror-debuginfo-2:17.2.7-1643.gd75f2382.el9.x86 1/1 2024-09-16T11:07:10.228 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: rbd-mirror-debuginfo-2:17.2.7-1643.gd75f2382.el9.x86 1/1 2024-09-16T11:07:10.457 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : rbd-mirror-debuginfo-2:17.2.7-1643.gd75f2382.el9.x86 1/1 2024-09-16T11:07:10.457 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-16T11:07:10.457 INFO:teuthology.orchestra.run.smithi102.stdout:Installed: 2024-09-16T11:07:10.457 INFO:teuthology.orchestra.run.smithi102.stdout: rbd-mirror-debuginfo-2:17.2.7-1643.gd75f2382.el9.x86_64 2024-09-16T11:07:10.457 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-16T11:07:10.457 INFO:teuthology.orchestra.run.smithi102.stdout:Complete! 2024-09-16T11:07:10.640 DEBUG:teuthology.orchestra.run.smithi102:> sudo yum -y install rbd-nbd-debuginfo 2024-09-16T11:07:11.202 INFO:teuthology.orchestra.run.smithi102.stdout:Last metadata expiration check: 0:08:01 ago on Mon 16 Sep 2024 10:59:10 AM UTC. 2024-09-16T11:07:11.335 INFO:teuthology.orchestra.run.smithi102.stdout:Dependencies resolved. 2024-09-16T11:07:11.335 INFO:teuthology.orchestra.run.smithi102.stdout:================================================================================ 2024-09-16T11:07:11.335 INFO:teuthology.orchestra.run.smithi102.stdout: Package Arch Version Repo Size 2024-09-16T11:07:11.335 INFO:teuthology.orchestra.run.smithi102.stdout:================================================================================ 2024-09-16T11:07:11.336 INFO:teuthology.orchestra.run.smithi102.stdout:Installing: 2024-09-16T11:07:11.336 INFO:teuthology.orchestra.run.smithi102.stdout: rbd-nbd-debuginfo x86_64 2:17.2.7-1643.gd75f2382.el9 ceph 1.2 M 2024-09-16T11:07:11.336 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-16T11:07:11.336 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction Summary 2024-09-16T11:07:11.336 INFO:teuthology.orchestra.run.smithi102.stdout:================================================================================ 2024-09-16T11:07:11.336 INFO:teuthology.orchestra.run.smithi102.stdout:Install 1 Package 2024-09-16T11:07:11.336 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-16T11:07:11.336 INFO:teuthology.orchestra.run.smithi102.stdout:Total download size: 1.2 M 2024-09-16T11:07:11.336 INFO:teuthology.orchestra.run.smithi102.stdout:Installed size: 5.4 M 2024-09-16T11:07:11.336 INFO:teuthology.orchestra.run.smithi102.stdout:Downloading Packages: 2024-09-16T11:07:11.681 INFO:teuthology.orchestra.run.smithi102.stdout:rbd-nbd-debuginfo-17.2.7-1643.gd75f2382.el9.x86 3.5 MB/s | 1.2 MB 00:00 2024-09-16T11:07:11.682 INFO:teuthology.orchestra.run.smithi102.stdout:-------------------------------------------------------------------------------- 2024-09-16T11:07:11.682 INFO:teuthology.orchestra.run.smithi102.stdout:Total 3.5 MB/s | 1.2 MB 00:00 2024-09-16T11:07:11.682 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction check 2024-09-16T11:07:11.685 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction check succeeded. 2024-09-16T11:07:11.685 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction test 2024-09-16T11:07:11.727 INFO:teuthology.orchestra.run.smithi102.stdout:Transaction test succeeded. 2024-09-16T11:07:11.727 INFO:teuthology.orchestra.run.smithi102.stdout:Running transaction 2024-09-16T11:07:11.955 INFO:teuthology.orchestra.run.smithi102.stdout: Preparing : 1/1 2024-09-16T11:07:12.053 INFO:teuthology.orchestra.run.smithi102.stdout: Installing : rbd-nbd-debuginfo-2:17.2.7-1643.gd75f2382.el9.x86_64 1/1 2024-09-16T11:07:12.451 INFO:teuthology.orchestra.run.smithi102.stdout: Running scriptlet: rbd-nbd-debuginfo-2:17.2.7-1643.gd75f2382.el9.x86_64 1/1 2024-09-16T11:07:12.706 INFO:teuthology.orchestra.run.smithi102.stdout: Verifying : rbd-nbd-debuginfo-2:17.2.7-1643.gd75f2382.el9.x86_64 1/1 2024-09-16T11:07:12.707 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-16T11:07:12.707 INFO:teuthology.orchestra.run.smithi102.stdout:Installed: 2024-09-16T11:07:12.707 INFO:teuthology.orchestra.run.smithi102.stdout: rbd-nbd-debuginfo-2:17.2.7-1643.gd75f2382.el9.x86_64 2024-09-16T11:07:12.707 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-16T11:07:12.707 INFO:teuthology.orchestra.run.smithi102.stdout:Complete! 2024-09-16T11:07:12.868 DEBUG:teuthology.parallel:result is None 2024-09-16T11:07:12.869 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=d75f23820806da622f561ee5ee7551015285993f 2024-09-16T11:07:13.010 DEBUG:teuthology.orchestra.run.smithi102:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-16T11:07:13.032 INFO:teuthology.orchestra.run.smithi102.stdout:17.2.7-1643.gd75f2382.el9 2024-09-16T11:07:13.032 INFO:teuthology.packaging:The installed version of ceph is 17.2.7-1643.gd75f2382.el9 2024-09-16T11:07:13.032 INFO:teuthology.task.install:The correct ceph version 17.2.7-1643.gd75f2382 is installed. 2024-09-16T11:07:13.034 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-09-16T11:07:13.034 DEBUG:teuthology.orchestra.run.smithi102:> set -ex 2024-09-16T11:07:13.035 DEBUG:teuthology.orchestra.run.smithi102:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-16T11:07:13.096 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-09-16T11:07:13.096 DEBUG:teuthology.orchestra.run.smithi102:> set -ex 2024-09-16T11:07:13.096 DEBUG:teuthology.orchestra.run.smithi102:> sudo dd of=/usr/bin/daemon-helper 2024-09-16T11:07:13.161 DEBUG:teuthology.orchestra.run.smithi102:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-16T11:07:13.226 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-09-16T11:07:13.226 DEBUG:teuthology.orchestra.run.smithi102:> set -ex 2024-09-16T11:07:13.226 DEBUG:teuthology.orchestra.run.smithi102:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-16T11:07:13.290 DEBUG:teuthology.orchestra.run.smithi102:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-16T11:07:13.355 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-09-16T11:07:13.355 DEBUG:teuthology.orchestra.run.smithi102:> set -ex 2024-09-16T11:07:13.355 DEBUG:teuthology.orchestra.run.smithi102:> sudo dd of=/usr/bin/stdin-killer 2024-09-16T11:07:13.420 DEBUG:teuthology.orchestra.run.smithi102:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-16T11:07:13.485 INFO:teuthology.run_tasks:Running task ceph... 2024-09-16T11:07:13.564 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-09-16T11:07:13.564 DEBUG:teuthology.orchestra.run.smithi102:> sudo chmod 777 /var/log/ceph 2024-09-16T11:07:13.589 INFO:tasks.ceph:Disabling ceph logrotate... 2024-09-16T11:07:13.589 DEBUG:teuthology.orchestra.run.smithi102:> sudo rm -f -- /etc/logrotate.d/ceph 2024-09-16T11:07:13.654 INFO:tasks.ceph:Creating extra log directories... 2024-09-16T11:07:13.654 DEBUG:teuthology.orchestra.run.smithi102:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-09-16T11:07:13.726 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-09-16T11:07:13.726 INFO:tasks.ceph:config {'conf': {'global': {'mon warn on pool no app': False}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'bluefs allocator': 'hybrid', 'bluestore allocator': 'hybrid', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd shutdown pgref assert': True}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-09-16T11:07:13.726 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/sangadi-2024-09-16_10:44:09-rbd-wip-bharath9-testing-2024-09-13-1444-quincy-distro-default-smithi/7907146', 'branch': 'wip-bharath9-testing-2024-09-13-1444-quincy', 'description': 'rbd/valgrind/{base/install centos_latest clusters/{fixed-1 openstack} conf/{disable-pool-app} objectstore/bluestore-hybrid validator/memcheck workloads/fsx}', 'email': 'sangadi@redhat.com', 'first_in_suite': False, 'job_id': '7907146', 'kernel': {'kdb': 1, 'sha1': 'distro'}, 'last_in_suite': False, 'machine_type': 'smithi', 'name': 'sangadi-2024-09-16_10:44:09-rbd-wip-bharath9-testing-2024-09-13-1444-quincy-distro-default-smithi', 'no_nested_subset': False, 'openstack': [{'volumes': {'count': 3, 'size': 30}}], 'os_type': 'centos', 'os_version': '9.stream', 'overrides': {'admin_socket': {'branch': 'wip-bharath9-testing-2024-09-13-1444-quincy'}, 'ceph': {'conf': {'global': {'mon warn on pool no app': False}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'bluefs allocator': 'hybrid', 'bluestore allocator': 'hybrid', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd shutdown pgref assert': True}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)'], 'sha1': 'd75f23820806da622f561ee5ee7551015285993f'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'global': {'osd crush chooseleaf type': 0, 'osd pool default pg num': 128, 'osd pool default pgp num': 128, 'osd pool default size': 2}, 'mon': {}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'debuginfo': True, 'enable_coprs': ['ceph/el9'], 'flavor': 'default', 'sha1': 'd75f23820806da622f561ee5ee7551015285993f'}}, 'rbd_fsx': {'valgrind': ['--tool=memcheck']}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-bharath9-testing-2024-09-13-1444-quincy', 'env': {'VALGRIND': '--tool=memcheck --leak-check=full'}, 'sha1': 'd75f23820806da622f561ee5ee7551015285993f'}}, 'owner': 'scheduled_sangadi@teuthology', 'priority': 130, 'repo': 'https://github.com/ceph/ceph-ci.git', 'roles': [['mon.a', 'mgr.x', 'osd.0', 'osd.1', 'osd.2', 'client.0']], 'seed': 9764, 'sha1': 'd75f23820806da622f561ee5ee7551015285993f', 'sleep_before_teardown': 0, 'subset': '111/120000', 'suite': 'rbd', 'suite_branch': 'wip-bharath9-testing-2024-09-13-1444-quincy', 'suite_path': '/home/teuthworker/src/github.com_ceph_ceph-c_d75f23820806da622f561ee5ee7551015285993f/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://github.com/ceph/ceph-ci.git', 'suite_sha1': 'd75f23820806da622f561ee5ee7551015285993f', 'targets': {'smithi102.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBM6iLK7rxqNnDZpwl7wO2SLuZFIQ92XBDe4yi74JjdzO70rvQqlPbil+kMjvlZVerdZ1ptWjMCq0rm2MQlJsDj8='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'kdb': 1, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': None}, {'ceph': None}, {'rbd_fsx': {'clients': ['client.0'], 'size': 134217728}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': []}, 'teuthology_branch': 'main', 'teuthology_sha1': '3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a', 'timestamp': '2024-09-16_10:44:09', 'tube': 'smithi', 'user': 'sangadi', 'verbose': True, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856'} 2024-09-16T11:07:13.727 DEBUG:teuthology.orchestra.run.smithi102:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-09-16T11:07:13.775 DEBUG:teuthology.orchestra.run.smithi102:> sudo install -d -m0777 -- /var/run/ceph 2024-09-16T11:07:13.841 DEBUG:teuthology.orchestra.run.smithi102:> set -ex 2024-09-16T11:07:13.841 DEBUG:teuthology.orchestra.run.smithi102:> dd if=/scratch_devs of=/dev/stdout 2024-09-16T11:07:13.896 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-16T11:07:13.897 DEBUG:teuthology.orchestra.run.smithi102:> stat /dev/vg_nvme/lv_1 2024-09-16T11:07:13.951 INFO:teuthology.orchestra.run.smithi102.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-16T11:07:13.952 INFO:teuthology.orchestra.run.smithi102.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-16T11:07:13.952 INFO:teuthology.orchestra.run.smithi102.stdout:Device: 5h/5d Inode: 667 Links: 1 2024-09-16T11:07:13.952 INFO:teuthology.orchestra.run.smithi102.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-16T11:07:13.952 INFO:teuthology.orchestra.run.smithi102.stdout:Context: system_u:object_r:device_t:s0 2024-09-16T11:07:13.952 INFO:teuthology.orchestra.run.smithi102.stdout:Access: 2024-09-16 11:00:52.921349979 +0000 2024-09-16T11:07:13.952 INFO:teuthology.orchestra.run.smithi102.stdout:Modify: 2024-09-16 10:58:31.981385158 +0000 2024-09-16T11:07:13.952 INFO:teuthology.orchestra.run.smithi102.stdout:Change: 2024-09-16 10:58:31.981385158 +0000 2024-09-16T11:07:13.952 INFO:teuthology.orchestra.run.smithi102.stdout: Birth: 2024-09-16 10:58:31.981385158 +0000 2024-09-16T11:07:13.952 DEBUG:teuthology.orchestra.run.smithi102:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-16T11:07:14.014 INFO:teuthology.orchestra.run.smithi102.stderr:1+0 records in 2024-09-16T11:07:14.015 INFO:teuthology.orchestra.run.smithi102.stderr:1+0 records out 2024-09-16T11:07:14.015 INFO:teuthology.orchestra.run.smithi102.stderr:512 bytes copied, 0.000182828 s, 2.8 MB/s 2024-09-16T11:07:14.016 DEBUG:teuthology.orchestra.run.smithi102:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-16T11:07:14.072 DEBUG:teuthology.orchestra.run.smithi102:> stat /dev/vg_nvme/lv_2 2024-09-16T11:07:14.127 INFO:teuthology.orchestra.run.smithi102.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-16T11:07:14.128 INFO:teuthology.orchestra.run.smithi102.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-16T11:07:14.128 INFO:teuthology.orchestra.run.smithi102.stdout:Device: 5h/5d Inode: 684 Links: 1 2024-09-16T11:07:14.128 INFO:teuthology.orchestra.run.smithi102.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-16T11:07:14.128 INFO:teuthology.orchestra.run.smithi102.stdout:Context: system_u:object_r:device_t:s0 2024-09-16T11:07:14.128 INFO:teuthology.orchestra.run.smithi102.stdout:Access: 2024-09-16 11:00:52.922349958 +0000 2024-09-16T11:07:14.128 INFO:teuthology.orchestra.run.smithi102.stdout:Modify: 2024-09-16 10:58:32.311378213 +0000 2024-09-16T11:07:14.128 INFO:teuthology.orchestra.run.smithi102.stdout:Change: 2024-09-16 10:58:32.311378213 +0000 2024-09-16T11:07:14.128 INFO:teuthology.orchestra.run.smithi102.stdout: Birth: 2024-09-16 10:58:32.311378213 +0000 2024-09-16T11:07:14.128 DEBUG:teuthology.orchestra.run.smithi102:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-16T11:07:14.191 INFO:teuthology.orchestra.run.smithi102.stderr:1+0 records in 2024-09-16T11:07:14.191 INFO:teuthology.orchestra.run.smithi102.stderr:1+0 records out 2024-09-16T11:07:14.191 INFO:teuthology.orchestra.run.smithi102.stderr:512 bytes copied, 0.000211639 s, 2.4 MB/s 2024-09-16T11:07:14.192 DEBUG:teuthology.orchestra.run.smithi102:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-16T11:07:14.249 DEBUG:teuthology.orchestra.run.smithi102:> stat /dev/vg_nvme/lv_3 2024-09-16T11:07:14.305 INFO:teuthology.orchestra.run.smithi102.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-16T11:07:14.305 INFO:teuthology.orchestra.run.smithi102.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-16T11:07:14.305 INFO:teuthology.orchestra.run.smithi102.stdout:Device: 5h/5d Inode: 705 Links: 1 2024-09-16T11:07:14.306 INFO:teuthology.orchestra.run.smithi102.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-16T11:07:14.306 INFO:teuthology.orchestra.run.smithi102.stdout:Context: system_u:object_r:device_t:s0 2024-09-16T11:07:14.306 INFO:teuthology.orchestra.run.smithi102.stdout:Access: 2024-09-16 11:00:52.922349958 +0000 2024-09-16T11:07:14.306 INFO:teuthology.orchestra.run.smithi102.stdout:Modify: 2024-09-16 10:58:32.632371457 +0000 2024-09-16T11:07:14.306 INFO:teuthology.orchestra.run.smithi102.stdout:Change: 2024-09-16 10:58:32.632371457 +0000 2024-09-16T11:07:14.306 INFO:teuthology.orchestra.run.smithi102.stdout: Birth: 2024-09-16 10:58:32.632371457 +0000 2024-09-16T11:07:14.306 DEBUG:teuthology.orchestra.run.smithi102:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-16T11:07:14.368 INFO:teuthology.orchestra.run.smithi102.stderr:1+0 records in 2024-09-16T11:07:14.369 INFO:teuthology.orchestra.run.smithi102.stderr:1+0 records out 2024-09-16T11:07:14.369 INFO:teuthology.orchestra.run.smithi102.stderr:512 bytes copied, 0.000188087 s, 2.7 MB/s 2024-09-16T11:07:14.370 DEBUG:teuthology.orchestra.run.smithi102:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-16T11:07:14.426 DEBUG:teuthology.orchestra.run.smithi102:> stat /dev/vg_nvme/lv_4 2024-09-16T11:07:14.481 INFO:teuthology.orchestra.run.smithi102.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-16T11:07:14.481 INFO:teuthology.orchestra.run.smithi102.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-16T11:07:14.481 INFO:teuthology.orchestra.run.smithi102.stdout:Device: 5h/5d Inode: 723 Links: 1 2024-09-16T11:07:14.481 INFO:teuthology.orchestra.run.smithi102.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-16T11:07:14.481 INFO:teuthology.orchestra.run.smithi102.stdout:Context: system_u:object_r:device_t:s0 2024-09-16T11:07:14.481 INFO:teuthology.orchestra.run.smithi102.stdout:Access: 2024-09-16 11:00:52.922349958 +0000 2024-09-16T11:07:14.481 INFO:teuthology.orchestra.run.smithi102.stdout:Modify: 2024-09-16 10:58:32.966364427 +0000 2024-09-16T11:07:14.481 INFO:teuthology.orchestra.run.smithi102.stdout:Change: 2024-09-16 10:58:32.966364427 +0000 2024-09-16T11:07:14.481 INFO:teuthology.orchestra.run.smithi102.stdout: Birth: 2024-09-16 10:58:32.966364427 +0000 2024-09-16T11:07:14.482 DEBUG:teuthology.orchestra.run.smithi102:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-16T11:07:14.544 INFO:teuthology.orchestra.run.smithi102.stderr:1+0 records in 2024-09-16T11:07:14.544 INFO:teuthology.orchestra.run.smithi102.stderr:1+0 records out 2024-09-16T11:07:14.544 INFO:teuthology.orchestra.run.smithi102.stderr:512 bytes copied, 0.000256408 s, 2.0 MB/s 2024-09-16T11:07:14.545 DEBUG:teuthology.orchestra.run.smithi102:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-16T11:07:14.602 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'} 2024-09-16T11:07:14.602 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi102.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'}} 2024-09-16T11:07:14.602 INFO:tasks.ceph:Generating config... 2024-09-16T11:07:14.603 INFO:tasks.ceph:[global] mon warn on pool no app = False 2024-09-16T11:07:14.603 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-09-16T11:07:14.603 INFO:tasks.ceph:[mgr] debug ms = 1 2024-09-16T11:07:14.604 INFO:tasks.ceph:[mon] debug mon = 20 2024-09-16T11:07:14.604 INFO:tasks.ceph:[mon] debug ms = 1 2024-09-16T11:07:14.604 INFO:tasks.ceph:[mon] debug paxos = 20 2024-09-16T11:07:14.604 INFO:tasks.ceph:[osd] bluefs allocator = hybrid 2024-09-16T11:07:14.604 INFO:tasks.ceph:[osd] bluestore allocator = hybrid 2024-09-16T11:07:14.604 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-09-16T11:07:14.604 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-09-16T11:07:14.604 INFO:tasks.ceph:[osd] debug bluefs = 1/20 2024-09-16T11:07:14.604 INFO:tasks.ceph:[osd] debug bluestore = 1/20 2024-09-16T11:07:14.604 INFO:tasks.ceph:[osd] debug ms = 1 2024-09-16T11:07:14.604 INFO:tasks.ceph:[osd] debug osd = 20 2024-09-16T11:07:14.604 INFO:tasks.ceph:[osd] debug rocksdb = 4/10 2024-09-16T11:07:14.604 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-09-16T11:07:14.604 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-09-16T11:07:14.605 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-09-16T11:07:14.605 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-09-16T11:07:14.605 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-09-16T11:07:14.605 INFO:tasks.ceph:[osd] osd shutdown pgref assert = True 2024-09-16T11:07:14.605 INFO:tasks.ceph:Setting up mon.a... 2024-09-16T11:07:14.605 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-09-16T11:07:14.690 INFO:teuthology.orchestra.run.smithi102.stdout:creating /etc/ceph/ceph.keyring 2024-09-16T11:07:14.693 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-09-16T11:07:14.731 DEBUG:teuthology.orchestra.run.smithi102:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-16T11:07:14.796 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.102')] 2024-09-16T11:07:14.797 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.102', 'mon warn on pool no app': False}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bluefs allocator': 'hybrid', 'bluestore allocator': 'hybrid', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd shutdown pgref assert': True}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false'}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok'}, 'mon.a': {}} 2024-09-16T11:07:14.797 DEBUG:teuthology.orchestra.run.smithi102:> set -ex 2024-09-16T11:07:14.798 DEBUG:teuthology.orchestra.run.smithi102:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-09-16T11:07:14.852 DEBUG:teuthology.orchestra.run.smithi102:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.102 --print /home/ubuntu/cephtest/ceph.monmap 2024-09-16T11:07:14.928 INFO:teuthology.orchestra.run.smithi102.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-09-16T11:07:14.928 INFO:teuthology.orchestra.run.smithi102.stdout:monmaptool: generated fsid 79f34890-6ac0-492b-826f-b29cf1045a98 2024-09-16T11:07:14.928 INFO:teuthology.orchestra.run.smithi102.stdout:setting min_mon_release = octopus 2024-09-16T11:07:14.928 INFO:teuthology.orchestra.run.smithi102.stdout:epoch 0 2024-09-16T11:07:14.928 INFO:teuthology.orchestra.run.smithi102.stdout:fsid 79f34890-6ac0-492b-826f-b29cf1045a98 2024-09-16T11:07:14.928 INFO:teuthology.orchestra.run.smithi102.stdout:last_changed 2024-09-16T11:07:14.927396+0000 2024-09-16T11:07:14.928 INFO:teuthology.orchestra.run.smithi102.stdout:created 2024-09-16T11:07:14.927396+0000 2024-09-16T11:07:14.928 INFO:teuthology.orchestra.run.smithi102.stdout:min_mon_release 15 (octopus) 2024-09-16T11:07:14.929 INFO:teuthology.orchestra.run.smithi102.stdout:election_strategy: 1 2024-09-16T11:07:14.929 INFO:teuthology.orchestra.run.smithi102.stdout:0: [v2:172.21.15.102:3300/0,v1:172.21.15.102:6789/0] mon.a 2024-09-16T11:07:14.929 INFO:teuthology.orchestra.run.smithi102.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (1 monitors) 2024-09-16T11:07:14.930 DEBUG:teuthology.orchestra.run.smithi102:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-09-16T11:07:14.985 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 79f34890-6ac0-492b-826f-b29cf1045a98... 2024-09-16T11:07:14.986 DEBUG:teuthology.orchestra.run.smithi102:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-16T11:07:15.067 INFO:teuthology.orchestra.run.smithi102.stdout:[global] 2024-09-16T11:07:15.067 INFO:teuthology.orchestra.run.smithi102.stdout: chdir = "" 2024-09-16T11:07:15.067 INFO:teuthology.orchestra.run.smithi102.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-16T11:07:15.067 INFO:teuthology.orchestra.run.smithi102.stdout: auth supported = cephx 2024-09-16T11:07:15.067 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-16T11:07:15.067 INFO:teuthology.orchestra.run.smithi102.stdout: filestore xattr use omap = true 2024-09-16T11:07:15.067 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-16T11:07:15.067 INFO:teuthology.orchestra.run.smithi102.stdout: mon clock drift allowed = 1.000 2024-09-16T11:07:15.068 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-16T11:07:15.068 INFO:teuthology.orchestra.run.smithi102.stdout: osd crush chooseleaf type = 0 2024-09-16T11:07:15.068 INFO:teuthology.orchestra.run.smithi102.stdout: auth debug = true 2024-09-16T11:07:15.068 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-16T11:07:15.068 INFO:teuthology.orchestra.run.smithi102.stdout: ms die on old message = true 2024-09-16T11:07:15.068 INFO:teuthology.orchestra.run.smithi102.stdout: ms die on bug = true 2024-09-16T11:07:15.068 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-16T11:07:15.068 INFO:teuthology.orchestra.run.smithi102.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-16T11:07:15.068 INFO:teuthology.orchestra.run.smithi102.stdout: mon pg warn max object skew = 0 2024-09-16T11:07:15.068 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-16T11:07:15.068 INFO:teuthology.orchestra.run.smithi102.stdout: # disable pg_autoscaler by default for new pools 2024-09-16T11:07:15.068 INFO:teuthology.orchestra.run.smithi102.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-16T11:07:15.068 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-16T11:07:15.068 INFO:teuthology.orchestra.run.smithi102.stdout: osd pool default size = 2 2024-09-16T11:07:15.069 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-16T11:07:15.069 INFO:teuthology.orchestra.run.smithi102.stdout: mon osd allow primary affinity = true 2024-09-16T11:07:15.069 INFO:teuthology.orchestra.run.smithi102.stdout: mon osd allow pg remap = true 2024-09-16T11:07:15.069 INFO:teuthology.orchestra.run.smithi102.stdout: mon warn on legacy crush tunables = false 2024-09-16T11:07:15.069 INFO:teuthology.orchestra.run.smithi102.stdout: mon warn on crush straw calc version zero = false 2024-09-16T11:07:15.069 INFO:teuthology.orchestra.run.smithi102.stdout: mon warn on no sortbitwise = false 2024-09-16T11:07:15.069 INFO:teuthology.orchestra.run.smithi102.stdout: mon warn on osd down out interval zero = false 2024-09-16T11:07:15.069 INFO:teuthology.orchestra.run.smithi102.stdout: mon warn on too few osds = false 2024-09-16T11:07:15.069 INFO:teuthology.orchestra.run.smithi102.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-16T11:07:15.069 INFO:teuthology.orchestra.run.smithi102.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-16T11:07:15.069 INFO:teuthology.orchestra.run.smithi102.stdout: mon_allow_pool_size_one = true 2024-09-16T11:07:15.069 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-16T11:07:15.069 INFO:teuthology.orchestra.run.smithi102.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-16T11:07:15.070 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-16T11:07:15.070 INFO:teuthology.orchestra.run.smithi102.stdout: osd default data pool replay window = 5 2024-09-16T11:07:15.070 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-16T11:07:15.070 INFO:teuthology.orchestra.run.smithi102.stdout: mon allow pool delete = true 2024-09-16T11:07:15.070 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-16T11:07:15.070 INFO:teuthology.orchestra.run.smithi102.stdout: mon cluster log file level = debug 2024-09-16T11:07:15.070 INFO:teuthology.orchestra.run.smithi102.stdout: debug asserts on shutdown = true 2024-09-16T11:07:15.070 INFO:teuthology.orchestra.run.smithi102.stdout: mon health detail to clog = false 2024-09-16T11:07:15.070 INFO:teuthology.orchestra.run.smithi102.stdout: mon host = 172.21.15.102 2024-09-16T11:07:15.070 INFO:teuthology.orchestra.run.smithi102.stdout: mon warn on pool no app = False 2024-09-16T11:07:15.070 INFO:teuthology.orchestra.run.smithi102.stdout: fsid = 79f34890-6ac0-492b-826f-b29cf1045a98 2024-09-16T11:07:15.070 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-16T11:07:15.070 INFO:teuthology.orchestra.run.smithi102.stdout:[osd] 2024-09-16T11:07:15.070 INFO:teuthology.orchestra.run.smithi102.stdout: osd journal size = 100 2024-09-16T11:07:15.070 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-16T11:07:15.071 INFO:teuthology.orchestra.run.smithi102.stdout: osd scrub load threshold = 5.0 2024-09-16T11:07:15.071 INFO:teuthology.orchestra.run.smithi102.stdout: osd scrub max interval = 600 2024-09-16T11:07:15.071 INFO:teuthology.orchestra.run.smithi102.stdout: osd mclock profile = high_recovery_ops 2024-09-16T11:07:15.071 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-16T11:07:15.071 INFO:teuthology.orchestra.run.smithi102.stdout: osd recover clone overlap = true 2024-09-16T11:07:15.071 INFO:teuthology.orchestra.run.smithi102.stdout: osd recovery max chunk = 1048576 2024-09-16T11:07:15.071 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-16T11:07:15.071 INFO:teuthology.orchestra.run.smithi102.stdout: osd debug shutdown = true 2024-09-16T11:07:15.071 INFO:teuthology.orchestra.run.smithi102.stdout: osd debug op order = true 2024-09-16T11:07:15.071 INFO:teuthology.orchestra.run.smithi102.stdout: osd debug verify stray on activate = true 2024-09-16T11:07:15.071 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-16T11:07:15.071 INFO:teuthology.orchestra.run.smithi102.stdout: osd open classes on start = true 2024-09-16T11:07:15.071 INFO:teuthology.orchestra.run.smithi102.stdout: osd debug pg log writeout = true 2024-09-16T11:07:15.071 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-16T11:07:15.072 INFO:teuthology.orchestra.run.smithi102.stdout: osd deep scrub update digest min age = 30 2024-09-16T11:07:15.072 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-16T11:07:15.072 INFO:teuthology.orchestra.run.smithi102.stdout: osd map max advance = 10 2024-09-16T11:07:15.072 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-16T11:07:15.072 INFO:teuthology.orchestra.run.smithi102.stdout: journal zero on create = true 2024-09-16T11:07:15.072 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-16T11:07:15.072 INFO:teuthology.orchestra.run.smithi102.stdout: filestore ondisk finisher threads = 3 2024-09-16T11:07:15.072 INFO:teuthology.orchestra.run.smithi102.stdout: filestore apply finisher threads = 3 2024-09-16T11:07:15.072 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-16T11:07:15.072 INFO:teuthology.orchestra.run.smithi102.stdout: bdev debug aio = true 2024-09-16T11:07:15.072 INFO:teuthology.orchestra.run.smithi102.stdout: osd debug misdirected ops = true 2024-09-16T11:07:15.072 INFO:teuthology.orchestra.run.smithi102.stdout: bluefs allocator = hybrid 2024-09-16T11:07:15.072 INFO:teuthology.orchestra.run.smithi102.stdout: bluestore allocator = hybrid 2024-09-16T11:07:15.072 INFO:teuthology.orchestra.run.smithi102.stdout: bluestore block size = 96636764160 2024-09-16T11:07:15.072 INFO:teuthology.orchestra.run.smithi102.stdout: bluestore fsck on mount = True 2024-09-16T11:07:15.073 INFO:teuthology.orchestra.run.smithi102.stdout: debug bluefs = 1/20 2024-09-16T11:07:15.073 INFO:teuthology.orchestra.run.smithi102.stdout: debug bluestore = 1/20 2024-09-16T11:07:15.073 INFO:teuthology.orchestra.run.smithi102.stdout: debug ms = 1 2024-09-16T11:07:15.073 INFO:teuthology.orchestra.run.smithi102.stdout: debug osd = 20 2024-09-16T11:07:15.073 INFO:teuthology.orchestra.run.smithi102.stdout: debug rocksdb = 4/10 2024-09-16T11:07:15.073 INFO:teuthology.orchestra.run.smithi102.stdout: mon osd backfillfull_ratio = 0.85 2024-09-16T11:07:15.073 INFO:teuthology.orchestra.run.smithi102.stdout: mon osd full ratio = 0.9 2024-09-16T11:07:15.073 INFO:teuthology.orchestra.run.smithi102.stdout: mon osd nearfull ratio = 0.8 2024-09-16T11:07:15.073 INFO:teuthology.orchestra.run.smithi102.stdout: osd failsafe full ratio = 0.95 2024-09-16T11:07:15.073 INFO:teuthology.orchestra.run.smithi102.stdout: osd objectstore = bluestore 2024-09-16T11:07:15.073 INFO:teuthology.orchestra.run.smithi102.stdout: osd shutdown pgref assert = True 2024-09-16T11:07:15.073 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-16T11:07:15.073 INFO:teuthology.orchestra.run.smithi102.stdout:[mgr] 2024-09-16T11:07:15.073 INFO:teuthology.orchestra.run.smithi102.stdout: debug ms = 1 2024-09-16T11:07:15.074 INFO:teuthology.orchestra.run.smithi102.stdout: debug mgr = 20 2024-09-16T11:07:15.074 INFO:teuthology.orchestra.run.smithi102.stdout: debug mon = 20 2024-09-16T11:07:15.074 INFO:teuthology.orchestra.run.smithi102.stdout: debug auth = 20 2024-09-16T11:07:15.074 INFO:teuthology.orchestra.run.smithi102.stdout: mon reweight min pgs per osd = 4 2024-09-16T11:07:15.074 INFO:teuthology.orchestra.run.smithi102.stdout: mon reweight min bytes per osd = 10 2024-09-16T11:07:15.074 INFO:teuthology.orchestra.run.smithi102.stdout: mgr/telemetry/nag = false 2024-09-16T11:07:15.074 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-16T11:07:15.074 INFO:teuthology.orchestra.run.smithi102.stdout:[mon] 2024-09-16T11:07:15.074 INFO:teuthology.orchestra.run.smithi102.stdout: debug ms = 1 2024-09-16T11:07:15.074 INFO:teuthology.orchestra.run.smithi102.stdout: debug mon = 20 2024-09-16T11:07:15.074 INFO:teuthology.orchestra.run.smithi102.stdout: debug paxos = 20 2024-09-16T11:07:15.074 INFO:teuthology.orchestra.run.smithi102.stdout: debug auth = 20 2024-09-16T11:07:15.074 INFO:teuthology.orchestra.run.smithi102.stdout: mon data avail warn = 5 2024-09-16T11:07:15.074 INFO:teuthology.orchestra.run.smithi102.stdout: mon mgr mkfs grace = 240 2024-09-16T11:07:15.074 INFO:teuthology.orchestra.run.smithi102.stdout: mon reweight min pgs per osd = 4 2024-09-16T11:07:15.075 INFO:teuthology.orchestra.run.smithi102.stdout: mon osd reporter subtree level = osd 2024-09-16T11:07:15.075 INFO:teuthology.orchestra.run.smithi102.stdout: mon osd prime pg temp = true 2024-09-16T11:07:15.075 INFO:teuthology.orchestra.run.smithi102.stdout: mon reweight min bytes per osd = 10 2024-09-16T11:07:15.075 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-16T11:07:15.075 INFO:teuthology.orchestra.run.smithi102.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-16T11:07:15.075 INFO:teuthology.orchestra.run.smithi102.stdout: auth mon ticket ttl = 660 # 11m 2024-09-16T11:07:15.075 INFO:teuthology.orchestra.run.smithi102.stdout: auth service ticket ttl = 240 # 4m 2024-09-16T11:07:15.075 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-16T11:07:15.075 INFO:teuthology.orchestra.run.smithi102.stdout: # don't complain about insecure global_id in the test suite 2024-09-16T11:07:15.075 INFO:teuthology.orchestra.run.smithi102.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-16T11:07:15.075 INFO:teuthology.orchestra.run.smithi102.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-16T11:07:15.075 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-16T11:07:15.075 INFO:teuthology.orchestra.run.smithi102.stdout: # 1m isn't quite enough 2024-09-16T11:07:15.075 INFO:teuthology.orchestra.run.smithi102.stdout: mon_down_mkfs_grace = 2m 2024-09-16T11:07:15.076 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-16T11:07:15.076 INFO:teuthology.orchestra.run.smithi102.stdout: mon_warn_on_filestore_osds = false 2024-09-16T11:07:15.076 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-16T11:07:15.076 INFO:teuthology.orchestra.run.smithi102.stdout:[client] 2024-09-16T11:07:15.076 INFO:teuthology.orchestra.run.smithi102.stdout: rgw cache enabled = true 2024-09-16T11:07:15.076 INFO:teuthology.orchestra.run.smithi102.stdout: rgw enable ops log = true 2024-09-16T11:07:15.076 INFO:teuthology.orchestra.run.smithi102.stdout: rgw enable usage log = true 2024-09-16T11:07:15.076 INFO:teuthology.orchestra.run.smithi102.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-16T11:07:15.076 INFO:teuthology.orchestra.run.smithi102.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-16T11:07:15.076 INFO:teuthology.orchestra.run.smithi102.stdout:[mon.a] 2024-09-16T11:07:15.078 INFO:tasks.ceph:Creating admin key on mon.a... 2024-09-16T11:07:15.079 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-09-16T11:07:15.168 INFO:tasks.ceph:Copying monmap to all nodes... 2024-09-16T11:07:15.168 DEBUG:teuthology.orchestra.run.smithi102:> set -ex 2024-09-16T11:07:15.168 DEBUG:teuthology.orchestra.run.smithi102:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-09-16T11:07:15.184 DEBUG:teuthology.orchestra.run.smithi102:> set -ex 2024-09-16T11:07:15.184 DEBUG:teuthology.orchestra.run.smithi102:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-09-16T11:07:15.238 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi102.front.sepia.ceph.com 2024-09-16T11:07:15.239 DEBUG:teuthology.orchestra.run.smithi102:> set -ex 2024-09-16T11:07:15.239 DEBUG:teuthology.orchestra.run.smithi102:> sudo dd of=/etc/ceph/ceph.keyring 2024-09-16T11:07:15.239 DEBUG:teuthology.orchestra.run.smithi102:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-16T11:07:15.313 DEBUG:teuthology.orchestra.run.smithi102:> set -ex 2024-09-16T11:07:15.313 DEBUG:teuthology.orchestra.run.smithi102:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-09-16T11:07:15.369 INFO:tasks.ceph:Setting up mon nodes... 2024-09-16T11:07:15.369 INFO:tasks.ceph:Setting up mgr nodes... 2024-09-16T11:07:15.369 DEBUG:teuthology.orchestra.run.smithi102:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-09-16T11:07:15.458 INFO:teuthology.orchestra.run.smithi102.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-09-16T11:07:15.460 INFO:tasks.ceph:Setting up mds nodes... 2024-09-16T11:07:15.460 INFO:tasks.ceph_client:Setting up client nodes... 2024-09-16T11:07:15.460 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-09-16T11:07:15.498 INFO:teuthology.orchestra.run.smithi102.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-09-16T11:07:15.510 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-09-16T11:07:15.510 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi102.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'}} 2024-09-16T11:07:15.510 DEBUG:teuthology.orchestra.run.smithi102:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-09-16T11:07:15.575 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'} 2024-09-16T11:07:15.575 INFO:tasks.ceph:role: osd.0 2024-09-16T11:07:15.575 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi102.front.sepia.ceph.com 2024-09-16T11:07:15.576 DEBUG:teuthology.orchestra.run.smithi102:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-09-16T11:07:15.648 INFO:teuthology.orchestra.run.smithi102.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-09-16T11:07:15.648 INFO:teuthology.orchestra.run.smithi102.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-16T11:07:15.648 INFO:teuthology.orchestra.run.smithi102.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-16T11:07:15.648 INFO:teuthology.orchestra.run.smithi102.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-16T11:07:15.648 INFO:teuthology.orchestra.run.smithi102.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-16T11:07:15.648 INFO:teuthology.orchestra.run.smithi102.stdout: = sunit=0 swidth=0 blks 2024-09-16T11:07:15.648 INFO:teuthology.orchestra.run.smithi102.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-16T11:07:15.648 INFO:teuthology.orchestra.run.smithi102.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-16T11:07:15.648 INFO:teuthology.orchestra.run.smithi102.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-16T11:07:15.649 INFO:teuthology.orchestra.run.smithi102.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-16T11:07:16.169 INFO:teuthology.orchestra.run.smithi102.stdout:Discarding blocks...Done. 2024-09-16T11:07:16.171 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi102.front.sepia.ceph.com -o noatime 2024-09-16T11:07:16.172 DEBUG:teuthology.orchestra.run.smithi102:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-09-16T11:07:16.278 DEBUG:teuthology.orchestra.run.smithi102:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-09-16T11:07:16.309 DEBUG:teuthology.orchestra.run.smithi102:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-09-16T11:07:16.375 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'} 2024-09-16T11:07:16.376 INFO:tasks.ceph:role: osd.1 2024-09-16T11:07:16.376 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi102.front.sepia.ceph.com 2024-09-16T11:07:16.376 DEBUG:teuthology.orchestra.run.smithi102:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-09-16T11:07:16.448 INFO:teuthology.orchestra.run.smithi102.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-09-16T11:07:16.448 INFO:teuthology.orchestra.run.smithi102.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-16T11:07:16.448 INFO:teuthology.orchestra.run.smithi102.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-16T11:07:16.448 INFO:teuthology.orchestra.run.smithi102.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-16T11:07:16.448 INFO:teuthology.orchestra.run.smithi102.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-16T11:07:16.448 INFO:teuthology.orchestra.run.smithi102.stdout: = sunit=0 swidth=0 blks 2024-09-16T11:07:16.448 INFO:teuthology.orchestra.run.smithi102.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-16T11:07:16.448 INFO:teuthology.orchestra.run.smithi102.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-16T11:07:16.448 INFO:teuthology.orchestra.run.smithi102.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-16T11:07:16.448 INFO:teuthology.orchestra.run.smithi102.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-16T11:07:16.910 INFO:teuthology.orchestra.run.smithi102.stdout:Discarding blocks...Done. 2024-09-16T11:07:16.912 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi102.front.sepia.ceph.com -o noatime 2024-09-16T11:07:16.912 DEBUG:teuthology.orchestra.run.smithi102:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-09-16T11:07:17.008 DEBUG:teuthology.orchestra.run.smithi102:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-09-16T11:07:17.037 DEBUG:teuthology.orchestra.run.smithi102:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-09-16T11:07:17.102 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'} 2024-09-16T11:07:17.102 INFO:tasks.ceph:role: osd.2 2024-09-16T11:07:17.102 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi102.front.sepia.ceph.com 2024-09-16T11:07:17.102 DEBUG:teuthology.orchestra.run.smithi102:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-09-16T11:07:17.174 INFO:teuthology.orchestra.run.smithi102.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-09-16T11:07:17.174 INFO:teuthology.orchestra.run.smithi102.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-16T11:07:17.174 INFO:teuthology.orchestra.run.smithi102.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-16T11:07:17.174 INFO:teuthology.orchestra.run.smithi102.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-16T11:07:17.174 INFO:teuthology.orchestra.run.smithi102.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-16T11:07:17.175 INFO:teuthology.orchestra.run.smithi102.stdout: = sunit=0 swidth=0 blks 2024-09-16T11:07:17.175 INFO:teuthology.orchestra.run.smithi102.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-16T11:07:17.175 INFO:teuthology.orchestra.run.smithi102.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-16T11:07:17.175 INFO:teuthology.orchestra.run.smithi102.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-16T11:07:17.175 INFO:teuthology.orchestra.run.smithi102.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-16T11:07:17.657 INFO:teuthology.orchestra.run.smithi102.stdout:Discarding blocks...Done. 2024-09-16T11:07:17.659 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi102.front.sepia.ceph.com -o noatime 2024-09-16T11:07:17.659 DEBUG:teuthology.orchestra.run.smithi102:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-09-16T11:07:17.809 DEBUG:teuthology.orchestra.run.smithi102:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-09-16T11:07:17.841 DEBUG:teuthology.orchestra.run.smithi102:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-16T11:07:17.935 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T11:07:17.933+0000 7fe41c11c440 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-09-16T11:07:17.937 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T11:07:17.935+0000 7fe41c11c440 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-09-16T11:07:17.937 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T11:07:17.935+0000 7fe41c11c440 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-09-16T11:07:17.937 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T11:07:17.935+0000 7fe41c11c440 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-09-16T11:07:17.937 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T11:07:17.935+0000 7fe41c11c440 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-09-16T11:07:17.943 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T11:07:17.942+0000 7fe41c11c440 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-09-16T11:07:18.474 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T11:07:18.472+0000 7fe41c11c440 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-09-16T11:07:19.235 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T11:07:19.233+0000 7fe41c11c440 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-09-16T11:07:19.997 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T11:07:19.995+0000 7fe41c11c440 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-09-16T11:07:20.791 DEBUG:teuthology.orchestra.run.smithi102:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-09-16T11:07:20.818 DEBUG:teuthology.orchestra.run.smithi102:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-16T11:07:20.902 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T11:07:20.900+0000 7f4c56050440 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-09-16T11:07:20.902 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T11:07:20.900+0000 7f4c56050440 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-09-16T11:07:20.903 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T11:07:20.901+0000 7f4c56050440 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-09-16T11:07:20.903 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T11:07:20.901+0000 7f4c56050440 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-09-16T11:07:20.903 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T11:07:20.901+0000 7f4c56050440 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-09-16T11:07:20.903 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T11:07:20.901+0000 7f4c56050440 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-09-16T11:07:21.426 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T11:07:21.424+0000 7f4c56050440 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-09-16T11:07:22.198 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T11:07:22.196+0000 7f4c56050440 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-09-16T11:07:22.961 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T11:07:22.959+0000 7f4c56050440 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-09-16T11:07:23.761 DEBUG:teuthology.orchestra.run.smithi102:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-09-16T11:07:23.827 DEBUG:teuthology.orchestra.run.smithi102:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-16T11:07:23.909 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T11:07:23.907+0000 7fd128d2c440 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-09-16T11:07:23.910 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T11:07:23.907+0000 7fd128d2c440 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-09-16T11:07:23.910 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T11:07:23.907+0000 7fd128d2c440 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-09-16T11:07:23.910 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T11:07:23.907+0000 7fd128d2c440 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-09-16T11:07:23.910 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T11:07:23.907+0000 7fd128d2c440 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-09-16T11:07:23.910 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T11:07:23.908+0000 7fd128d2c440 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-09-16T11:07:24.434 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T11:07:24.433+0000 7fd128d2c440 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-09-16T11:07:25.209 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T11:07:25.207+0000 7fd128d2c440 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-09-16T11:07:25.973 INFO:teuthology.orchestra.run.smithi102.stderr:2024-09-16T11:07:25.971+0000 7fd128d2c440 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-09-16T11:07:26.769 DEBUG:teuthology.orchestra.run.smithi102:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-09-16T11:07:26.838 INFO:tasks.ceph:Reading keys from all nodes... 2024-09-16T11:07:26.838 DEBUG:teuthology.orchestra.run.smithi102:> set -ex 2024-09-16T11:07:26.838 DEBUG:teuthology.orchestra.run.smithi102:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-09-16T11:07:26.905 DEBUG:teuthology.orchestra.run.smithi102:> set -ex 2024-09-16T11:07:26.906 DEBUG:teuthology.orchestra.run.smithi102:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-09-16T11:07:26.973 DEBUG:teuthology.orchestra.run.smithi102:> set -ex 2024-09-16T11:07:26.973 DEBUG:teuthology.orchestra.run.smithi102:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-09-16T11:07:27.039 DEBUG:teuthology.orchestra.run.smithi102:> set -ex 2024-09-16T11:07:27.039 DEBUG:teuthology.orchestra.run.smithi102:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-09-16T11:07:27.106 DEBUG:teuthology.orchestra.run.smithi102:> set -ex 2024-09-16T11:07:27.106 DEBUG:teuthology.orchestra.run.smithi102:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-09-16T11:07:27.162 INFO:tasks.ceph:Adding keys to all mons... 2024-09-16T11:07:27.163 DEBUG:teuthology.orchestra.run.smithi102:> sudo tee -a /etc/ceph/ceph.keyring 2024-09-16T11:07:27.226 INFO:teuthology.orchestra.run.smithi102.stdout:[mgr.x] 2024-09-16T11:07:27.226 INFO:teuthology.orchestra.run.smithi102.stdout: key = AQBjEehmQhJBGxAAi2GYoMiSFHEt87nXfMMqVA== 2024-09-16T11:07:27.227 INFO:teuthology.orchestra.run.smithi102.stdout:[osd.0] 2024-09-16T11:07:27.227 INFO:teuthology.orchestra.run.smithi102.stdout: key = AQBlEehmDrezNxAAoL/BGGqYjHoPf0lpo2+TcQ== 2024-09-16T11:07:27.227 INFO:teuthology.orchestra.run.smithi102.stdout:[osd.1] 2024-09-16T11:07:27.227 INFO:teuthology.orchestra.run.smithi102.stdout: key = AQBoEehmqbG9NRAABOm290vFK72X+tP9QbYTIQ== 2024-09-16T11:07:27.227 INFO:teuthology.orchestra.run.smithi102.stdout:[osd.2] 2024-09-16T11:07:27.227 INFO:teuthology.orchestra.run.smithi102.stdout: key = AQBrEehmYugnNhAANeWq/jxpRTr+mjjGpcY0zg== 2024-09-16T11:07:27.227 INFO:teuthology.orchestra.run.smithi102.stdout:[client.0] 2024-09-16T11:07:27.227 INFO:teuthology.orchestra.run.smithi102.stdout: key = AQBjEehmvGWoHRAA1jUNHNaYutGonG/NJ4EM7w== 2024-09-16T11:07:27.228 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-16T11:07:27.315 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-16T11:07:27.361 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-16T11:07:27.407 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-16T11:07:27.452 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-16T11:07:27.500 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-09-16T11:07:27.501 DEBUG:teuthology.orchestra.run.smithi102:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-09-16T11:07:27.525 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-16T11:07:27.615 DEBUG:teuthology.orchestra.run.smithi102:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-09-16T11:07:27.639 DEBUG:teuthology.orchestra.run.smithi102:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-09-16T11:07:27.694 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-09-16T11:07:27.695 INFO:tasks.ceph.mon.a:Restarting daemon 2024-09-16T11:07:27.695 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-09-16T11:07:27.737 INFO:tasks.ceph.mon.a:Started 2024-09-16T11:07:27.737 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-09-16T11:07:27.737 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-09-16T11:07:27.737 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-09-16T11:07:27.740 INFO:tasks.ceph.mgr.x:Started 2024-09-16T11:07:27.740 INFO:tasks.ceph:Setting crush tunables to default 2024-09-16T11:07:27.740 DEBUG:teuthology.orchestra.run.smithi102:> sudo ceph --cluster ceph osd crush tunables default 2024-09-16T11:07:27.901 INFO:teuthology.orchestra.run.smithi102.stderr:adjusted tunables profile to default 2024-09-16T11:07:27.911 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-09-16T11:07:27.911 DEBUG:teuthology.orchestra.run.smithi102:> set -ex 2024-09-16T11:07:27.911 DEBUG:teuthology.orchestra.run.smithi102:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-09-16T11:07:27.935 DEBUG:teuthology.orchestra.run.smithi102:> set -ex 2024-09-16T11:07:27.935 DEBUG:teuthology.orchestra.run.smithi102:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-09-16T11:07:27.999 DEBUG:teuthology.orchestra.run.smithi102:> set -ex 2024-09-16T11:07:27.999 DEBUG:teuthology.orchestra.run.smithi102:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-09-16T11:07:28.064 DEBUG:teuthology.orchestra.run.smithi102:> sudo ceph --cluster ceph osd new 73775a88-4914-46f2-a066-b4f012c226c5 0 2024-09-16T11:07:28.114 INFO:tasks.ceph.mgr.x.smithi102.stderr:2024-09-16T11:07:28.113+0000 7fadaaa0fec0 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-16T11:07:28.156 INFO:tasks.ceph.mgr.x.smithi102.stderr:2024-09-16T11:07:28.154+0000 7fadaaa0fec0 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-16T11:07:28.211 INFO:tasks.ceph.mgr.x.smithi102.stderr:2024-09-16T11:07:28.209+0000 7fadaaa0fec0 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-16T11:07:28.285 INFO:teuthology.orchestra.run.smithi102.stdout:0 2024-09-16T11:07:28.296 DEBUG:teuthology.orchestra.run.smithi102:> sudo ceph --cluster ceph osd new 566bbdc4-ed6a-4580-a04b-7d4fa97fcc81 1 2024-09-16T11:07:28.366 INFO:tasks.ceph.mgr.x.smithi102.stderr:2024-09-16T11:07:28.364+0000 7fadaaa0fec0 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-16T11:07:28.477 INFO:tasks.ceph.mgr.x.smithi102.stderr:2024-09-16T11:07:28.475+0000 7fadaaa0fec0 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-16T11:07:28.505 INFO:teuthology.orchestra.run.smithi102.stdout:1 2024-09-16T11:07:28.516 DEBUG:teuthology.orchestra.run.smithi102:> sudo ceph --cluster ceph osd new 4aaa7267-412c-4e0f-b6f9-dd70e45cdc8a 2 2024-09-16T11:07:28.725 INFO:teuthology.orchestra.run.smithi102.stdout:2 2024-09-16T11:07:28.736 INFO:tasks.ceph.osd.0:Restarting daemon 2024-09-16T11:07:28.736 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-09-16T11:07:28.738 INFO:tasks.ceph.osd.0:Started 2024-09-16T11:07:28.738 INFO:tasks.ceph.osd.1:Restarting daemon 2024-09-16T11:07:28.738 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-09-16T11:07:28.740 INFO:tasks.ceph.osd.1:Started 2024-09-16T11:07:28.740 INFO:tasks.ceph.osd.2:Restarting daemon 2024-09-16T11:07:28.740 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-09-16T11:07:28.742 INFO:tasks.ceph.osd.2:Started 2024-09-16T11:07:28.742 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-16T11:07:28.777 INFO:tasks.ceph.mgr.x.smithi102.stderr:2024-09-16T11:07:28.776+0000 7fadaaa0fec0 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-16T11:07:28.840 INFO:tasks.ceph.mgr.x.smithi102.stderr:2024-09-16T11:07:28.839+0000 7fadaaa0fec0 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-16T11:07:28.920 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-16T11:07:28.920 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":5,"fsid":"79f34890-6ac0-492b-826f-b29cf1045a98","created":"2024-09-16T11:07:27.825113+0000","modified":"2024-09-16T11:07:28.722914+0000","last_up_change":"0.000000","last_in_change":"2024-09-16T11:07:28.722914+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[],"osds":[{"osd":0,"uuid":"73775a88-4914-46f2-a066-b4f012c226c5","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"566bbdc4-ed6a-4580-a04b-7d4fa97fcc81","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"4aaa7267-412c-4e0f-b6f9-dd70e45cdc8a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T11:07:28.930 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-09-16T11:07:28.931 INFO:tasks.ceph:Waiting for OSDs to come up 2024-09-16T11:07:28.977 INFO:tasks.ceph.mgr.x.smithi102.stderr:2024-09-16T11:07:28.975+0000 7fadaaa0fec0 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-16T11:07:29.232 DEBUG:teuthology.orchestra.run.smithi102:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-16T11:07:29.330 INFO:tasks.ceph.osd.0.smithi102.stderr:2024-09-16T11:07:29.328+0000 7f66e268d440 -1 Falling back to public interface 2024-09-16T11:07:29.333 INFO:tasks.ceph.osd.2.smithi102.stderr:2024-09-16T11:07:29.331+0000 7f0d2a034440 -1 Falling back to public interface 2024-09-16T11:07:29.345 INFO:tasks.ceph.osd.1.smithi102.stderr:2024-09-16T11:07:29.343+0000 7fa099240440 -1 Falling back to public interface 2024-09-16T11:07:29.390 INFO:teuthology.misc.health.smithi102.stdout: 2024-09-16T11:07:29.390 INFO:teuthology.misc.health.smithi102.stdout:{"epoch":5,"fsid":"79f34890-6ac0-492b-826f-b29cf1045a98","created":"2024-09-16T11:07:27.825113+0000","modified":"2024-09-16T11:07:28.722914+0000","last_up_change":"0.000000","last_in_change":"2024-09-16T11:07:28.722914+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[],"osds":[{"osd":0,"uuid":"73775a88-4914-46f2-a066-b4f012c226c5","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"566bbdc4-ed6a-4580-a04b-7d4fa97fcc81","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"4aaa7267-412c-4e0f-b6f9-dd70e45cdc8a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T11:07:29.399 DEBUG:teuthology.misc:0 of 3 OSDs are up 2024-09-16T11:07:29.889 INFO:tasks.ceph.mgr.x.smithi102.stderr:2024-09-16T11:07:29.888+0000 7fadaaa0fec0 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-16T11:07:29.939 INFO:tasks.ceph.mgr.x.smithi102.stderr:2024-09-16T11:07:29.937+0000 7fadaaa0fec0 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-16T11:07:30.299 INFO:tasks.ceph.mgr.x.smithi102.stderr:2024-09-16T11:07:30.297+0000 7fadaaa0fec0 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-16T11:07:30.378 INFO:tasks.ceph.mgr.x.smithi102.stderr:2024-09-16T11:07:30.376+0000 7fadaaa0fec0 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-16T11:07:30.467 INFO:tasks.ceph.mgr.x.smithi102.stderr:2024-09-16T11:07:30.465+0000 7fadaaa0fec0 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-16T11:07:30.511 INFO:tasks.ceph.mgr.x.smithi102.stderr:2024-09-16T11:07:30.509+0000 7fadaaa0fec0 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-16T11:07:30.602 INFO:tasks.ceph.mgr.x.smithi102.stderr:2024-09-16T11:07:30.601+0000 7fadaaa0fec0 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-16T11:07:30.653 INFO:tasks.ceph.mgr.x.smithi102.stderr:2024-09-16T11:07:30.652+0000 7fadaaa0fec0 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-16T11:07:30.696 INFO:tasks.ceph.mgr.x.smithi102.stderr:2024-09-16T11:07:30.693+0000 7fadaaa0fec0 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-16T11:07:30.789 INFO:tasks.ceph.mgr.x.smithi102.stderr:2024-09-16T11:07:30.786+0000 7fadaaa0fec0 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-16T11:07:30.837 INFO:tasks.ceph.mgr.x.smithi102.stderr:2024-09-16T11:07:30.834+0000 7fadaaa0fec0 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-16T11:07:30.885 INFO:tasks.ceph.mgr.x.smithi102.stderr:2024-09-16T11:07:30.883+0000 7fadaaa0fec0 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-16T11:07:30.950 INFO:tasks.ceph.mgr.x.smithi102.stderr:2024-09-16T11:07:30.947+0000 7fadaaa0fec0 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-16T11:07:30.992 INFO:tasks.ceph.mgr.x.smithi102.stderr:2024-09-16T11:07:30.990+0000 7fadaaa0fec0 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-16T11:07:31.036 INFO:tasks.ceph.mgr.x.smithi102.stderr:2024-09-16T11:07:31.033+0000 7fadaaa0fec0 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-16T11:07:31.643 INFO:tasks.ceph.mgr.x.smithi102.stderr:2024-09-16T11:07:31.640+0000 7fadaaa0fec0 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-16T11:07:32.650 INFO:tasks.ceph.mgr.x.smithi102.stderr:2024-09-16T11:07:32.648+0000 7fad8f689640 -1 mgr.server handle_report got status from non-daemon mon.a 2024-09-16T11:07:32.848 INFO:tasks.ceph.osd.0.smithi102.stderr:2024-09-16T11:07:32.846+0000 7f66e268d440 -1 osd.0 0 log_to_monitors true 2024-09-16T11:07:32.884 INFO:tasks.ceph.osd.2.smithi102.stderr:2024-09-16T11:07:32.883+0000 7f0d2a034440 -1 osd.2 0 log_to_monitors true 2024-09-16T11:07:32.887 INFO:tasks.ceph.osd.1.smithi102.stderr:2024-09-16T11:07:32.886+0000 7fa099240440 -1 osd.1 0 log_to_monitors true 2024-09-16T11:07:35.702 DEBUG:teuthology.orchestra.run.smithi102:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-16T11:07:36.110 INFO:teuthology.misc.health.smithi102.stdout: 2024-09-16T11:07:36.110 INFO:teuthology.misc.health.smithi102.stdout:{"epoch":7,"fsid":"79f34890-6ac0-492b-826f-b29cf1045a98","created":"2024-09-16T11:07:27.825113+0000","modified":"2024-09-16T11:07:34.653914+0000","last_up_change":"0.000000","last_in_change":"2024-09-16T11:07:28.722914+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[],"osds":[{"osd":0,"uuid":"73775a88-4914-46f2-a066-b4f012c226c5","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"566bbdc4-ed6a-4580-a04b-7d4fa97fcc81","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"4aaa7267-412c-4e0f-b6f9-dd70e45cdc8a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T11:07:36.124 DEBUG:teuthology.misc:0 of 3 OSDs are up 2024-09-16T11:07:36.410 INFO:tasks.ceph.osd.2.smithi102.stderr:2024-09-16T11:07:36.409+0000 7f0d251f6640 -1 osd.2 0 waiting for initial osdmap 2024-09-16T11:07:36.416 INFO:tasks.ceph.osd.2.smithi102.stderr:2024-09-16T11:07:36.414+0000 7f0d1e084640 -1 osd.2 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-16T11:07:36.431 INFO:tasks.ceph.osd.0.smithi102.stderr:2024-09-16T11:07:36.429+0000 7f66dd7f6640 -1 osd.0 0 waiting for initial osdmap 2024-09-16T11:07:36.435 INFO:tasks.ceph.osd.0.smithi102.stderr:2024-09-16T11:07:36.434+0000 7f66d67e8640 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-16T11:07:36.475 INFO:tasks.ceph.osd.1.smithi102.stderr:2024-09-16T11:07:36.473+0000 7fa0939f6640 -1 osd.1 0 waiting for initial osdmap 2024-09-16T11:07:36.479 INFO:tasks.ceph.osd.1.smithi102.stderr:2024-09-16T11:07:36.478+0000 7fa08c9e8640 -1 osd.1 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-16T11:07:42.427 DEBUG:teuthology.orchestra.run.smithi102:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-16T11:07:42.689 INFO:teuthology.misc.health.smithi102.stdout: 2024-09-16T11:07:42.689 INFO:teuthology.misc.health.smithi102.stdout:{"epoch":11,"fsid":"79f34890-6ac0-492b-826f-b29cf1045a98","created":"2024-09-16T11:07:27.825113+0000","modified":"2024-09-16T11:07:40.417889+0000","last_up_change":"2024-09-16T11:07:37.411085+0000","last_in_change":"2024-09-16T11:07:28.722914+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T11:07:37.676106+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"11","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"73775a88-4914-46f2-a066-b4f012c226c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6800","nonce":32395},{"type":"v1","addr":"172.21.15.102:6801","nonce":32395}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6802","nonce":32395},{"type":"v1","addr":"172.21.15.102:6803","nonce":32395}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6806","nonce":32395},{"type":"v1","addr":"172.21.15.102:6807","nonce":32395}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6804","nonce":32395},{"type":"v1","addr":"172.21.15.102:6805","nonce":32395}]},"public_addr":"172.21.15.102:6801/32395","cluster_addr":"172.21.15.102:6803/32395","heartbeat_back_addr":"172.21.15.102:6807/32395","heartbeat_front_addr":"172.21.15.102:6805/32395","state":["exists","up"]},{"osd":1,"uuid":"566bbdc4-ed6a-4580-a04b-7d4fa97fcc81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":9,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6816","nonce":32397},{"type":"v1","addr":"172.21.15.102:6817","nonce":32397}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6818","nonce":32397},{"type":"v1","addr":"172.21.15.102:6819","nonce":32397}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6822","nonce":32397},{"type":"v1","addr":"172.21.15.102:6823","nonce":32397}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6820","nonce":32397},{"type":"v1","addr":"172.21.15.102:6821","nonce":32397}]},"public_addr":"172.21.15.102:6817/32397","cluster_addr":"172.21.15.102:6819/32397","heartbeat_back_addr":"172.21.15.102:6823/32397","heartbeat_front_addr":"172.21.15.102:6821/32397","state":["exists","up"]},{"osd":2,"uuid":"4aaa7267-412c-4e0f-b6f9-dd70e45cdc8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6808","nonce":32396},{"type":"v1","addr":"172.21.15.102:6809","nonce":32396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6810","nonce":32396},{"type":"v1","addr":"172.21.15.102:6811","nonce":32396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6814","nonce":32396},{"type":"v1","addr":"172.21.15.102:6815","nonce":32396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6812","nonce":32396},{"type":"v1","addr":"172.21.15.102:6813","nonce":32396}]},"public_addr":"172.21.15.102:6809/32396","cluster_addr":"172.21.15.102:6811/32396","heartbeat_back_addr":"172.21.15.102:6815/32396","heartbeat_front_addr":"172.21.15.102:6813/32396","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T11:07:33.892765+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T11:07:33.888914+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T11:07:33.917416+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T11:07:42.700 DEBUG:teuthology.misc:3 of 3 OSDs are up 2024-09-16T11:07:42.700 INFO:tasks.ceph:Creating RBD pool 2024-09-16T11:07:42.701 DEBUG:teuthology.orchestra.run.smithi102:> sudo ceph --cluster ceph osd pool create rbd 8 2024-09-16T11:07:43.438 INFO:teuthology.orchestra.run.smithi102.stderr:pool 'rbd' created 2024-09-16T11:07:43.454 DEBUG:teuthology.orchestra.run.smithi102:> rbd --cluster ceph pool init rbd 2024-09-16T11:07:46.456 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-09-16T11:07:46.457 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-09-16T11:07:46.457 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-09-16T11:07:46.457 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-09-16T11:07:46.458 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-09-16T11:07:46.768 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-16T11:07:46.779 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":5,"flags":0,"active_gid":4102,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6824","nonce":32059},{"type":"v1","addr":"172.21.15.102:6825","nonce":32059}]},"active_addr":"172.21.15.102:6825/32059","active_change":"2024-09-16T11:07:31.643407+0000","active_mgr_features":4540138320759226367,"available":true,"standbys":[],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.1.5","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_requests":{"name":"max_requests","type":"int","level":"advanced","flags":0,"default_value":"500","min":"","max":"","enum_allowed":[],"desc":"Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number.if un-finished request is removed, error message will be logged in the ceph-mgr log.","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.102:0","nonce":3478964115}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.102:0","nonce":265175831}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.102:0","nonce":2147576711}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.102:0","nonce":1941242166}]}]} 2024-09-16T11:07:46.781 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-09-16T11:07:46.781 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-09-16T11:07:46.781 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-16T11:07:47.036 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-16T11:07:47.037 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":15,"fsid":"79f34890-6ac0-492b-826f-b29cf1045a98","created":"2024-09-16T11:07:27.825113+0000","modified":"2024-09-16T11:07:46.443515+0000","last_up_change":"2024-09-16T11:07:37.411085+0000","last_in_change":"2024-09-16T11:07:28.722914+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T11:07:37.676106+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"11","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-16T11:07:42.972213+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":15,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"73775a88-4914-46f2-a066-b4f012c226c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":12,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6800","nonce":32395},{"type":"v1","addr":"172.21.15.102:6801","nonce":32395}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6802","nonce":32395},{"type":"v1","addr":"172.21.15.102:6803","nonce":32395}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6806","nonce":32395},{"type":"v1","addr":"172.21.15.102:6807","nonce":32395}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6804","nonce":32395},{"type":"v1","addr":"172.21.15.102:6805","nonce":32395}]},"public_addr":"172.21.15.102:6801/32395","cluster_addr":"172.21.15.102:6803/32395","heartbeat_back_addr":"172.21.15.102:6807/32395","heartbeat_front_addr":"172.21.15.102:6805/32395","state":["exists","up"]},{"osd":1,"uuid":"566bbdc4-ed6a-4580-a04b-7d4fa97fcc81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":12,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6816","nonce":32397},{"type":"v1","addr":"172.21.15.102:6817","nonce":32397}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6818","nonce":32397},{"type":"v1","addr":"172.21.15.102:6819","nonce":32397}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6822","nonce":32397},{"type":"v1","addr":"172.21.15.102:6823","nonce":32397}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6820","nonce":32397},{"type":"v1","addr":"172.21.15.102:6821","nonce":32397}]},"public_addr":"172.21.15.102:6817/32397","cluster_addr":"172.21.15.102:6819/32397","heartbeat_back_addr":"172.21.15.102:6823/32397","heartbeat_front_addr":"172.21.15.102:6821/32397","state":["exists","up"]},{"osd":2,"uuid":"4aaa7267-412c-4e0f-b6f9-dd70e45cdc8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":12,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6808","nonce":32396},{"type":"v1","addr":"172.21.15.102:6809","nonce":32396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6810","nonce":32396},{"type":"v1","addr":"172.21.15.102:6811","nonce":32396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6814","nonce":32396},{"type":"v1","addr":"172.21.15.102:6815","nonce":32396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6812","nonce":32396},{"type":"v1","addr":"172.21.15.102:6813","nonce":32396}]},"public_addr":"172.21.15.102:6809/32396","cluster_addr":"172.21.15.102:6811/32396","heartbeat_back_addr":"172.21.15.102:6815/32396","heartbeat_front_addr":"172.21.15.102:6813/32396","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T11:07:33.892765+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T11:07:33.888914+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T11:07:33.917416+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_removed_snaps":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T11:07:47.048 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-09-16T11:07:47.049 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-16T11:07:47.305 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-16T11:07:47.305 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":15,"fsid":"79f34890-6ac0-492b-826f-b29cf1045a98","created":"2024-09-16T11:07:27.825113+0000","modified":"2024-09-16T11:07:46.443515+0000","last_up_change":"2024-09-16T11:07:37.411085+0000","last_in_change":"2024-09-16T11:07:28.722914+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T11:07:37.676106+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"11","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-16T11:07:42.972213+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":15,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"73775a88-4914-46f2-a066-b4f012c226c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":12,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6800","nonce":32395},{"type":"v1","addr":"172.21.15.102:6801","nonce":32395}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6802","nonce":32395},{"type":"v1","addr":"172.21.15.102:6803","nonce":32395}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6806","nonce":32395},{"type":"v1","addr":"172.21.15.102:6807","nonce":32395}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6804","nonce":32395},{"type":"v1","addr":"172.21.15.102:6805","nonce":32395}]},"public_addr":"172.21.15.102:6801/32395","cluster_addr":"172.21.15.102:6803/32395","heartbeat_back_addr":"172.21.15.102:6807/32395","heartbeat_front_addr":"172.21.15.102:6805/32395","state":["exists","up"]},{"osd":1,"uuid":"566bbdc4-ed6a-4580-a04b-7d4fa97fcc81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":12,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6816","nonce":32397},{"type":"v1","addr":"172.21.15.102:6817","nonce":32397}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6818","nonce":32397},{"type":"v1","addr":"172.21.15.102:6819","nonce":32397}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6822","nonce":32397},{"type":"v1","addr":"172.21.15.102:6823","nonce":32397}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6820","nonce":32397},{"type":"v1","addr":"172.21.15.102:6821","nonce":32397}]},"public_addr":"172.21.15.102:6817/32397","cluster_addr":"172.21.15.102:6819/32397","heartbeat_back_addr":"172.21.15.102:6823/32397","heartbeat_front_addr":"172.21.15.102:6821/32397","state":["exists","up"]},{"osd":2,"uuid":"4aaa7267-412c-4e0f-b6f9-dd70e45cdc8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":12,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6808","nonce":32396},{"type":"v1","addr":"172.21.15.102:6809","nonce":32396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6810","nonce":32396},{"type":"v1","addr":"172.21.15.102:6811","nonce":32396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6814","nonce":32396},{"type":"v1","addr":"172.21.15.102:6815","nonce":32396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6812","nonce":32396},{"type":"v1","addr":"172.21.15.102:6813","nonce":32396}]},"public_addr":"172.21.15.102:6809/32396","cluster_addr":"172.21.15.102:6811/32396","heartbeat_back_addr":"172.21.15.102:6815/32396","heartbeat_front_addr":"172.21.15.102:6813/32396","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T11:07:33.892765+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T11:07:33.888914+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T11:07:33.917416+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_removed_snaps":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T11:07:47.318 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-09-16T11:07:47.318 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-09-16T11:07:47.319 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-09-16T11:07:47.448 INFO:teuthology.orchestra.run.smithi102.stdout:34359738373 2024-09-16T11:07:47.448 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-16T11:07:47.464 INFO:teuthology.orchestra.run.smithi102.stdout:34359738374 2024-09-16T11:07:47.465 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-16T11:07:47.465 INFO:teuthology.orchestra.run.smithi102.stdout:34359738373 2024-09-16T11:07:47.465 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-16T11:07:47.777 INFO:teuthology.orchestra.run.smithi102.stdout:34359738372 2024-09-16T11:07:47.777 INFO:teuthology.orchestra.run.smithi102.stdout:34359738372 2024-09-16T11:07:47.791 INFO:tasks.ceph.ceph_manager.ceph:need seq 34359738373 got 34359738372 for osd.2 2024-09-16T11:07:47.792 INFO:tasks.ceph.ceph_manager.ceph:need seq 34359738373 got 34359738372 for osd.0 2024-09-16T11:07:47.825 INFO:teuthology.orchestra.run.smithi102.stdout:34359738373 2024-09-16T11:07:47.837 INFO:tasks.ceph.ceph_manager.ceph:need seq 34359738374 got 34359738373 for osd.1 2024-09-16T11:07:48.792 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-16T11:07:48.793 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-16T11:07:48.838 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-16T11:07:49.081 INFO:teuthology.orchestra.run.smithi102.stdout:34359738373 2024-09-16T11:07:49.083 INFO:teuthology.orchestra.run.smithi102.stdout:34359738373 2024-09-16T11:07:49.094 INFO:tasks.ceph.ceph_manager.ceph:need seq 34359738373 got 34359738373 for osd.2 2024-09-16T11:07:49.095 DEBUG:teuthology.parallel:result is None 2024-09-16T11:07:49.096 INFO:tasks.ceph.ceph_manager.ceph:need seq 34359738373 got 34359738373 for osd.0 2024-09-16T11:07:49.096 DEBUG:teuthology.parallel:result is None 2024-09-16T11:07:49.119 INFO:teuthology.orchestra.run.smithi102.stdout:34359738374 2024-09-16T11:07:49.132 INFO:tasks.ceph.ceph_manager.ceph:need seq 34359738374 got 34359738374 for osd.1 2024-09-16T11:07:49.132 DEBUG:teuthology.parallel:result is None 2024-09-16T11:07:49.132 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-16T11:07:49.132 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T11:07:49.427 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-16T11:07:49.428 INFO:teuthology.orchestra.run.smithi102.stderr:dumped all 2024-09-16T11:07:49.439 INFO:teuthology.orchestra.run.smithi102.stdout:{"pg_ready":true,"pg_map":{"version":18,"stamp":"2024-09-16T11:07:47.649244+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":56,"num_read_kb":46,"num_write":81,"num_write_kb":1108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":55,"ondisk_log_size":55,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":4,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":893600,"kb_used_data":1528,"kb_used_omap":4,"kb_used_meta":892027,"kb_avail":282221920,"statfs":{"total":289910292480,"available":288995246080,"internally_reserved":0,"allocated":1564672,"data_stored":1286761,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":5112,"internal_metadata":913435656},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.212532"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":14,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-16T11:07:46.454335+0000","last_change":"2024-09-16T11:07:46.454524+0000","last_active":"2024-09-16T11:07:46.454335+0000","last_peered":"2024-09-16T11:07:46.454335+0000","last_clean":"2024-09-16T11:07:46.454335+0000","last_became_active":"2024-09-16T11:07:44.446646+0000","last_became_peered":"2024-09-16T11:07:44.446646+0000","last_unstale":"2024-09-16T11:07:46.454335+0000","last_undegraded":"2024-09-16T11:07:46.454335+0000","last_fullsized":"2024-09-16T11:07:46.454335+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:07:43.427920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:07:43.427920+0000","last_clean_scrub_stamp":"2024-09-16T11:07:43.427920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:34:38.920978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000337344,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":14,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-16T11:07:46.454596+0000","last_change":"2024-09-16T11:07:46.454822+0000","last_active":"2024-09-16T11:07:46.454596+0000","last_peered":"2024-09-16T11:07:46.454596+0000","last_clean":"2024-09-16T11:07:46.454596+0000","last_became_active":"2024-09-16T11:07:44.446406+0000","last_became_peered":"2024-09-16T11:07:44.446406+0000","last_unstale":"2024-09-16T11:07:46.454596+0000","last_undegraded":"2024-09-16T11:07:46.454596+0000","last_fullsized":"2024-09-16T11:07:46.454596+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:07:43.427920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:07:43.427920+0000","last_clean_scrub_stamp":"2024-09-16T11:07:43.427920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:12:45.650225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062301399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-16T11:07:46.454832+0000","last_change":"2024-09-16T11:07:46.455055+0000","last_active":"2024-09-16T11:07:46.454832+0000","last_peered":"2024-09-16T11:07:46.454832+0000","last_clean":"2024-09-16T11:07:46.454832+0000","last_became_active":"2024-09-16T11:07:44.446498+0000","last_became_peered":"2024-09-16T11:07:44.446498+0000","last_unstale":"2024-09-16T11:07:46.454832+0000","last_undegraded":"2024-09-16T11:07:46.454832+0000","last_fullsized":"2024-09-16T11:07:46.454832+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:07:43.427920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:07:43.427920+0000","last_clean_scrub_stamp":"2024-09-16T11:07:43.427920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:40:08.726582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00084251400000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-16T11:07:46.454562+0000","last_change":"2024-09-16T11:07:46.454774+0000","last_active":"2024-09-16T11:07:46.454562+0000","last_peered":"2024-09-16T11:07:46.454562+0000","last_clean":"2024-09-16T11:07:46.454562+0000","last_became_active":"2024-09-16T11:07:44.446413+0000","last_became_peered":"2024-09-16T11:07:44.446413+0000","last_unstale":"2024-09-16T11:07:46.454562+0000","last_undegraded":"2024-09-16T11:07:46.454562+0000","last_fullsized":"2024-09-16T11:07:46.454562+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:07:43.427920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:07:43.427920+0000","last_clean_scrub_stamp":"2024-09-16T11:07:43.427920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:53:07.204424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059042800000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.2","version":"15'2","reported_seq":16,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-16T11:07:46.448909+0000","last_change":"2024-09-16T11:07:46.448909+0000","last_active":"2024-09-16T11:07:46.448909+0000","last_peered":"2024-09-16T11:07:46.448909+0000","last_clean":"2024-09-16T11:07:46.448909+0000","last_became_active":"2024-09-16T11:07:44.449069+0000","last_became_peered":"2024-09-16T11:07:44.449069+0000","last_unstale":"2024-09-16T11:07:46.448909+0000","last_undegraded":"2024-09-16T11:07:46.448909+0000","last_fullsized":"2024-09-16T11:07:46.448909+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:07:43.427920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:07:43.427920+0000","last_clean_scrub_stamp":"2024-09-16T11:07:43.427920+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:13:44.990400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00029305100000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":13,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-16T11:07:46.720126+0000","last_change":"2024-09-16T11:07:46.720289+0000","last_active":"2024-09-16T11:07:46.720126+0000","last_peered":"2024-09-16T11:07:46.720126+0000","last_clean":"2024-09-16T11:07:46.720126+0000","last_became_active":"2024-09-16T11:07:44.447533+0000","last_became_peered":"2024-09-16T11:07:44.447533+0000","last_unstale":"2024-09-16T11:07:46.720126+0000","last_undegraded":"2024-09-16T11:07:46.720126+0000","last_fullsized":"2024-09-16T11:07:46.720126+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:07:43.427920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:07:43.427920+0000","last_clean_scrub_stamp":"2024-09-16T11:07:43.427920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:13:44.990400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000230378,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":13,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-16T11:07:46.720091+0000","last_change":"2024-09-16T11:07:46.720264+0000","last_active":"2024-09-16T11:07:46.720091+0000","last_peered":"2024-09-16T11:07:46.720091+0000","last_clean":"2024-09-16T11:07:46.720091+0000","last_became_active":"2024-09-16T11:07:44.448153+0000","last_became_peered":"2024-09-16T11:07:44.448153+0000","last_unstale":"2024-09-16T11:07:46.720091+0000","last_undegraded":"2024-09-16T11:07:46.720091+0000","last_fullsized":"2024-09-16T11:07:46.720091+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:07:43.427920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:07:43.427920+0000","last_clean_scrub_stamp":"2024-09-16T11:07:43.427920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:09:42.536129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00024792500000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.3","version":"13'1","reported_seq":15,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-16T11:07:46.454301+0000","last_change":"2024-09-16T11:07:46.454516+0000","last_active":"2024-09-16T11:07:46.454301+0000","last_peered":"2024-09-16T11:07:46.454301+0000","last_clean":"2024-09-16T11:07:46.454301+0000","last_became_active":"2024-09-16T11:07:44.446367+0000","last_became_peered":"2024-09-16T11:07:44.446367+0000","last_unstale":"2024-09-16T11:07:46.454301+0000","last_undegraded":"2024-09-16T11:07:46.454301+0000","last_fullsized":"2024-09-16T11:07:46.454301+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:07:43.427920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:07:43.427920+0000","last_clean_scrub_stamp":"2024-09-16T11:07:43.427920+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:06:21.202391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00038771200000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"10'52","reported_seq":80,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-09-16T11:07:45.445397+0000","last_change":"2024-09-16T11:07:39.420884+0000","last_active":"2024-09-16T11:07:45.445397+0000","last_peered":"2024-09-16T11:07:45.445397+0000","last_clean":"2024-09-16T11:07:45.445397+0000","last_became_active":"2024-09-16T11:07:39.420564+0000","last_became_peered":"2024-09-16T11:07:39.420564+0000","last_unstale":"2024-09-16T11:07:45.445397+0000","last_undegraded":"2024-09-16T11:07:45.445397+0000","last_fullsized":"2024-09-16T11:07:45.445397+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:07:38.411971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:07:38.411971+0000","last_clean_scrub_stamp":"2024-09-16T11:07:38.411971+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:52:50.808244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":56,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":56,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":52,"ondisk_log_size":52,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359738373,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297468,"kb_used_data":108,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074372,"statfs":{"total":96636764160,"available":96332156928,"internally_reserved":0,"allocated":110592,"data_stored":30398,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28499999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32800000000000001}]}]},{"osd":1,"up_from":8,"seq":34359738374,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298068,"kb_used_data":712,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94073772,"statfs":{"total":96636764160,"available":96331542528,"internally_reserved":0,"allocated":729088,"data_stored":628562,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.20399999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.216}]}]},{"osd":0,"up_from":8,"seq":34359738373,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298064,"kb_used_data":708,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94073776,"statfs":{"total":96636764160,"available":96331546624,"internally_reserved":0,"allocated":724992,"data_stored":627801,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.24099999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.20799999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T11:07:49.440 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T11:07:49.696 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-16T11:07:49.697 INFO:teuthology.orchestra.run.smithi102.stderr:dumped all 2024-09-16T11:07:49.708 INFO:teuthology.orchestra.run.smithi102.stdout:{"pg_ready":true,"pg_map":{"version":19,"stamp":"2024-09-16T11:07:49.649588+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":56,"num_read_kb":46,"num_write":81,"num_write_kb":1108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":55,"ondisk_log_size":55,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":4,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":893600,"kb_used_data":1528,"kb_used_omap":4,"kb_used_meta":892027,"kb_avail":282221920,"statfs":{"total":289910292480,"available":288995246080,"internally_reserved":0,"allocated":1564672,"data_stored":1286761,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":5112,"internal_metadata":913435656},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001053"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":14,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-16T11:07:46.454335+0000","last_change":"2024-09-16T11:07:46.454524+0000","last_active":"2024-09-16T11:07:46.454335+0000","last_peered":"2024-09-16T11:07:46.454335+0000","last_clean":"2024-09-16T11:07:46.454335+0000","last_became_active":"2024-09-16T11:07:44.446646+0000","last_became_peered":"2024-09-16T11:07:44.446646+0000","last_unstale":"2024-09-16T11:07:46.454335+0000","last_undegraded":"2024-09-16T11:07:46.454335+0000","last_fullsized":"2024-09-16T11:07:46.454335+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:07:43.427920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:07:43.427920+0000","last_clean_scrub_stamp":"2024-09-16T11:07:43.427920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:34:38.920978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000337344,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":14,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-16T11:07:46.454596+0000","last_change":"2024-09-16T11:07:46.454822+0000","last_active":"2024-09-16T11:07:46.454596+0000","last_peered":"2024-09-16T11:07:46.454596+0000","last_clean":"2024-09-16T11:07:46.454596+0000","last_became_active":"2024-09-16T11:07:44.446406+0000","last_became_peered":"2024-09-16T11:07:44.446406+0000","last_unstale":"2024-09-16T11:07:46.454596+0000","last_undegraded":"2024-09-16T11:07:46.454596+0000","last_fullsized":"2024-09-16T11:07:46.454596+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:07:43.427920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:07:43.427920+0000","last_clean_scrub_stamp":"2024-09-16T11:07:43.427920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:12:45.650225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062301399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-16T11:07:46.454832+0000","last_change":"2024-09-16T11:07:46.455055+0000","last_active":"2024-09-16T11:07:46.454832+0000","last_peered":"2024-09-16T11:07:46.454832+0000","last_clean":"2024-09-16T11:07:46.454832+0000","last_became_active":"2024-09-16T11:07:44.446498+0000","last_became_peered":"2024-09-16T11:07:44.446498+0000","last_unstale":"2024-09-16T11:07:46.454832+0000","last_undegraded":"2024-09-16T11:07:46.454832+0000","last_fullsized":"2024-09-16T11:07:46.454832+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:07:43.427920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:07:43.427920+0000","last_clean_scrub_stamp":"2024-09-16T11:07:43.427920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:40:08.726582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00084251400000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-16T11:07:46.454562+0000","last_change":"2024-09-16T11:07:46.454774+0000","last_active":"2024-09-16T11:07:46.454562+0000","last_peered":"2024-09-16T11:07:46.454562+0000","last_clean":"2024-09-16T11:07:46.454562+0000","last_became_active":"2024-09-16T11:07:44.446413+0000","last_became_peered":"2024-09-16T11:07:44.446413+0000","last_unstale":"2024-09-16T11:07:46.454562+0000","last_undegraded":"2024-09-16T11:07:46.454562+0000","last_fullsized":"2024-09-16T11:07:46.454562+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:07:43.427920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:07:43.427920+0000","last_clean_scrub_stamp":"2024-09-16T11:07:43.427920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:53:07.204424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059042800000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.2","version":"15'2","reported_seq":16,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-16T11:07:46.448909+0000","last_change":"2024-09-16T11:07:46.448909+0000","last_active":"2024-09-16T11:07:46.448909+0000","last_peered":"2024-09-16T11:07:46.448909+0000","last_clean":"2024-09-16T11:07:46.448909+0000","last_became_active":"2024-09-16T11:07:44.449069+0000","last_became_peered":"2024-09-16T11:07:44.449069+0000","last_unstale":"2024-09-16T11:07:46.448909+0000","last_undegraded":"2024-09-16T11:07:46.448909+0000","last_fullsized":"2024-09-16T11:07:46.448909+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:07:43.427920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:07:43.427920+0000","last_clean_scrub_stamp":"2024-09-16T11:07:43.427920+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:13:44.990400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00029305100000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":13,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-16T11:07:46.720126+0000","last_change":"2024-09-16T11:07:46.720289+0000","last_active":"2024-09-16T11:07:46.720126+0000","last_peered":"2024-09-16T11:07:46.720126+0000","last_clean":"2024-09-16T11:07:46.720126+0000","last_became_active":"2024-09-16T11:07:44.447533+0000","last_became_peered":"2024-09-16T11:07:44.447533+0000","last_unstale":"2024-09-16T11:07:46.720126+0000","last_undegraded":"2024-09-16T11:07:46.720126+0000","last_fullsized":"2024-09-16T11:07:46.720126+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:07:43.427920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:07:43.427920+0000","last_clean_scrub_stamp":"2024-09-16T11:07:43.427920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:13:44.990400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000230378,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":13,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-16T11:07:46.720091+0000","last_change":"2024-09-16T11:07:46.720264+0000","last_active":"2024-09-16T11:07:46.720091+0000","last_peered":"2024-09-16T11:07:46.720091+0000","last_clean":"2024-09-16T11:07:46.720091+0000","last_became_active":"2024-09-16T11:07:44.448153+0000","last_became_peered":"2024-09-16T11:07:44.448153+0000","last_unstale":"2024-09-16T11:07:46.720091+0000","last_undegraded":"2024-09-16T11:07:46.720091+0000","last_fullsized":"2024-09-16T11:07:46.720091+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:07:43.427920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:07:43.427920+0000","last_clean_scrub_stamp":"2024-09-16T11:07:43.427920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:09:42.536129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00024792500000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"2.3","version":"13'1","reported_seq":15,"reported_epoch":15,"state":"active+clean","last_fresh":"2024-09-16T11:07:46.454301+0000","last_change":"2024-09-16T11:07:46.454516+0000","last_active":"2024-09-16T11:07:46.454301+0000","last_peered":"2024-09-16T11:07:46.454301+0000","last_clean":"2024-09-16T11:07:46.454301+0000","last_became_active":"2024-09-16T11:07:44.446367+0000","last_became_peered":"2024-09-16T11:07:44.446367+0000","last_unstale":"2024-09-16T11:07:46.454301+0000","last_undegraded":"2024-09-16T11:07:46.454301+0000","last_fullsized":"2024-09-16T11:07:46.454301+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:07:43.427920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:07:43.427920+0000","last_clean_scrub_stamp":"2024-09-16T11:07:43.427920+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:06:21.202391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00038771200000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"10'52","reported_seq":80,"reported_epoch":14,"state":"active+clean","last_fresh":"2024-09-16T11:07:45.445397+0000","last_change":"2024-09-16T11:07:39.420884+0000","last_active":"2024-09-16T11:07:45.445397+0000","last_peered":"2024-09-16T11:07:45.445397+0000","last_clean":"2024-09-16T11:07:45.445397+0000","last_became_active":"2024-09-16T11:07:39.420564+0000","last_became_peered":"2024-09-16T11:07:39.420564+0000","last_unstale":"2024-09-16T11:07:45.445397+0000","last_undegraded":"2024-09-16T11:07:45.445397+0000","last_fullsized":"2024-09-16T11:07:45.445397+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:07:38.411971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:07:38.411971+0000","last_clean_scrub_stamp":"2024-09-16T11:07:38.411971+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:52:50.808244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":56,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":56,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":52,"ondisk_log_size":52,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359738374,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297468,"kb_used_data":108,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074372,"statfs":{"total":96636764160,"available":96332156928,"internally_reserved":0,"allocated":110592,"data_stored":30398,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28499999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32800000000000001}]}]},{"osd":1,"up_from":8,"seq":34359738375,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298068,"kb_used_data":712,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94073772,"statfs":{"total":96636764160,"available":96331542528,"internally_reserved":0,"allocated":729088,"data_stored":628562,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.20399999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.216}]}]},{"osd":0,"up_from":8,"seq":34359738374,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298064,"kb_used_data":708,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94073776,"statfs":{"total":96636764160,"available":96331546624,"internally_reserved":0,"allocated":724992,"data_stored":627801,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.24099999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.20799999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T11:07:49.709 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-16T11:07:49.710 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-09-16T11:07:49.710 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-09-16T11:07:49.710 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-09-16T11:07:49.997 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-16T11:07:49.997 INFO:teuthology.orchestra.run.smithi102.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-09-16T11:07:50.009 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-09-16T11:07:50.009 INFO:teuthology.run_tasks:Running task rbd_fsx... 2024-09-16T11:07:50.022 INFO:tasks.rbd_fsx:starting rbd_fsx... 2024-09-16T11:07:50.022 DEBUG:tasks.ceph_manager:running fsx_client.0 under valgrind with args ['cd', '/home/ubuntu/cephtest', Raw('&&'), 'adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'env', 'OPENSSL_ia32cap=~0x1000000000000000', 'valgrind', '--trace-children=no', '--child-silent-after-fork=yes', '--soname-synonyms=somalloc=*tcmalloc*', '--num-callers=50', '--suppressions=/home/ubuntu/cephtest/valgrind.supp', '--xml=yes', '--xml-file=/var/log/ceph/valgrind/fsx_client.0.log', '--time-stamp=yes', '--vgdb=yes', '--exit-on-first-error=yes', '--error-exitcode=42', '--tool=memcheck'] 2024-09-16T11:07:50.023 DEBUG:teuthology.orchestra.run.smithi102:> cd /home/ubuntu/cephtest && adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage env 'OPENSSL_ia32cap=~0x1000000000000000' valgrind --trace-children=no --child-silent-after-fork=yes '--soname-synonyms=somalloc=*tcmalloc*' --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/fsx_client.0.log --time-stamp=yes --vgdb=yes --exit-on-first-error=yes --error-exitcode=42 --tool=memcheck ceph_test_librbd_fsx --cluster ceph --id 0 -d -W -R -p 100 -P /home/ubuntu/cephtest/archive -r 1 -w 1 -t 1 -h 1 -l 134217728 -S 0 -N 1000 pool_client.0 image_client.0 2024-09-16T11:08:02.027 INFO:teuthology.orchestra.run.smithi102.stdout:mapped writes DISABLED 2024-09-16T11:08:02.028 INFO:teuthology.orchestra.run.smithi102.stdout:mapped reads DISABLED 2024-09-16T11:08:02.035 INFO:teuthology.orchestra.run.smithi102.stdout:Seed set to 1069 2024-09-16T11:08:19.195 INFO:teuthology.orchestra.run.smithi102.stdout:skipping zero size read 2024-09-16T11:08:19.196 INFO:teuthology.orchestra.run.smithi102.stdout:skipping zero size read 2024-09-16T11:08:19.247 INFO:teuthology.orchestra.run.smithi102.stdout:3 writesame 0x3c0703d thru 0x3c15633 data_size 0x1(0xe5f7 bytes) 2024-09-16T11:08:19.283 INFO:teuthology.orchestra.run.smithi102.stdout:4 read 0x4298bc thru 0x434caa (0xb3ef bytes) 2024-09-16T11:08:19.315 INFO:teuthology.orchestra.run.smithi102.stdout:truncating to largest ever: 0x5d4349 2024-09-16T11:08:19.338 INFO:teuthology.orchestra.run.smithi102.stdout:7 trunc from 0x3c15634 to 0x5d4349 2024-09-16T11:08:19.384 INFO:teuthology.orchestra.run.smithi102.stdout:9 writesame 0x8f4ee2 thru 0x8fff11 data_size 0x8(0xb030 bytes) 2024-09-16T11:08:19.385 INFO:teuthology.orchestra.run.smithi102.stdout:14 read 0x59e7d2 thru 0x5a30ff (0x492e bytes) 2024-09-16T11:08:19.385 INFO:teuthology.orchestra.run.smithi102.stdout:15 read 0x23368c thru 0x23d8d7 (0xa24c bytes) 2024-09-16T11:08:19.386 INFO:teuthology.orchestra.run.smithi102.stdout:16 read 0x366179 thru 0x367bcc (0x1a54 bytes) 2024-09-16T11:08:19.387 INFO:teuthology.orchestra.run.smithi102.stdout:truncating to largest ever: 0x1e8d7ea 2024-09-16T11:08:19.396 INFO:teuthology.orchestra.run.smithi102.stdout:17 trunc from 0x8fff12 to 0x1e8d7ea 2024-09-16T11:08:19.457 INFO:teuthology.orchestra.run.smithi102.stdout:18 writesame 0x670efa4 thru 0x6711b18 data_size 0x19(0x2b75 bytes) 2024-09-16T11:08:19.459 INFO:teuthology.orchestra.run.smithi102.stdout:19 punch from 0xb9443 to 0xc39e9, (0xa5a6 bytes) 2024-09-16T11:08:19.467 INFO:teuthology.orchestra.run.smithi102.stdout:20 writesame 0x1084965 thru 0x108b5fd data_size 0x3(0x6c99 bytes) 2024-09-16T11:08:19.551 INFO:teuthology.orchestra.run.smithi102.stdout:21 punch from 0x6bc83a to 0x6ca8b2, (0xe078 bytes) 2024-09-16T11:08:19.554 INFO:teuthology.orchestra.run.smithi102.stdout:22 write 0x13abc30 thru 0x13b9ce2 (0xe0b3 bytes) 2024-09-16T11:08:19.583 INFO:teuthology.orchestra.run.smithi102.stdout:24 read 0x5d98c45 thru 0x5d99d33 (0x10ef bytes) 2024-09-16T11:08:19.584 INFO:teuthology.orchestra.run.smithi102.stdout:compare and write disabled 2024-09-16T11:08:19.584 INFO:teuthology.orchestra.run.smithi102.stdout:truncating to largest ever: 0x3616959 2024-09-16T11:08:19.607 INFO:teuthology.orchestra.run.smithi102.stdout:26 trunc from 0x6711b19 to 0x3616959 2024-09-16T11:08:19.641 INFO:teuthology.orchestra.run.smithi102.stdout:27 write 0x3d71bdb thru 0x3d760fd (0x4523 bytes) 2024-09-16T11:08:19.647 INFO:teuthology.orchestra.run.smithi102.stdout:28 punch from 0xc23a96 to 0xc26853, (0x2dbd bytes) 2024-09-16T11:08:19.647 INFO:teuthology.orchestra.run.smithi102.stdout:30 punch from 0x1640de8 to 0x1646d32, (0x5f4a bytes) 2024-09-16T11:08:19.679 INFO:teuthology.orchestra.run.smithi102.stdout:31 writesame 0x655054d thru 0x65505c8 data_size 0x2(0x7c bytes) 2024-09-16T11:08:19.679 INFO:teuthology.orchestra.run.smithi102.stdout:32 read 0x9b302e thru 0x9b776a (0x473d bytes) 2024-09-16T11:08:19.710 INFO:teuthology.orchestra.run.smithi102.stdout:33 trunc from 0x65505c9 to 0x1fc3468 2024-09-16T11:08:19.740 INFO:teuthology.orchestra.run.smithi102.stdout:35 write 0x1965192 thru 0x196f424 (0xa293 bytes) 2024-09-16T11:08:19.745 INFO:teuthology.orchestra.run.smithi102.stdout:truncating to largest ever: 0x50684c5 2024-09-16T11:08:19.766 INFO:teuthology.orchestra.run.smithi102.stdout:36 trunc from 0x1fc3468 to 0x50684c5 2024-09-16T11:08:19.780 INFO:teuthology.orchestra.run.smithi102.stdout:37 write 0x2301a53 thru 0x230994c (0x7efa bytes) 2024-09-16T11:08:19.785 INFO:teuthology.orchestra.run.smithi102.stdout:compare and write disabled 2024-09-16T11:08:19.804 INFO:teuthology.orchestra.run.smithi102.stdout:39 writesame 0x564f898 thru 0x5651dbc data_size 0x1(0x2525 bytes) 2024-09-16T11:08:19.814 INFO:teuthology.orchestra.run.smithi102.stdout:41 trunc from 0x5651dbd to 0x3f8cc66 2024-09-16T11:08:19.852 INFO:teuthology.orchestra.run.smithi102.stdout:42 write 0x52c8188 thru 0x52c9930 (0x17a9 bytes) 2024-09-16T11:08:19.857 INFO:teuthology.orchestra.run.smithi102.stdout:44 read 0x80b235 thru 0x81a71f (0xf4eb bytes) 2024-09-16T11:08:19.858 INFO:teuthology.orchestra.run.smithi102.stdout:46 read 0x46b53c4 thru 0x46c2c44 (0xd881 bytes) 2024-09-16T11:08:19.859 INFO:teuthology.orchestra.run.smithi102.stdout:47 read 0xe684d3 thru 0xe7336d (0xae9b bytes) 2024-09-16T11:08:19.886 INFO:teuthology.orchestra.run.smithi102.stdout:48 write 0x6b72257 thru 0x6b79a05 (0x77af bytes) 2024-09-16T11:08:19.932 INFO:teuthology.orchestra.run.smithi102.stdout:49 trunc from 0x6b79a06 to 0xd75642 2024-09-16T11:08:19.958 INFO:teuthology.orchestra.run.smithi102.stdout:50 read 0x1ae3de thru 0x1b5f8f (0x7bb2 bytes) 2024-09-16T11:08:19.974 INFO:teuthology.orchestra.run.smithi102.stdout:51 writesame 0xea03f6 thru 0xea708e data_size 0x3(0x6c99 bytes) 2024-09-16T11:08:19.975 INFO:teuthology.orchestra.run.smithi102.stdout:54 read 0x690dd0 thru 0x6956ae (0x48df bytes) 2024-09-16T11:08:19.975 INFO:teuthology.orchestra.run.smithi102.stdout:55 punch from 0x9b1455 to 0x9b8671, (0x721c bytes) 2024-09-16T11:08:19.976 INFO:teuthology.orchestra.run.smithi102.stdout:56 read 0x1164fc thru 0x118444 (0x1f49 bytes) 2024-09-16T11:08:20.005 INFO:teuthology.orchestra.run.smithi102.stdout:57 writesame 0x2e794d2 thru 0x2e846c5 data_size 0xe(0xb1f4 bytes) 2024-09-16T11:08:20.005 INFO:teuthology.orchestra.run.smithi102.stdout:compare and write disabled 2024-09-16T11:08:20.005 INFO:teuthology.orchestra.run.smithi102.stdout:61 read 0x1eaf46e thru 0x1ebeed2 (0xfa65 bytes) 2024-09-16T11:08:20.006 INFO:teuthology.orchestra.run.smithi102.stdout:62 punch from 0x2bb29f9 to 0x2bbaf3e, (0x8545 bytes) 2024-09-16T11:08:20.007 INFO:teuthology.orchestra.run.smithi102.stdout:63 read 0xa247b8 thru 0xa34491 (0xfcda bytes) 2024-09-16T11:08:20.009 INFO:teuthology.orchestra.run.smithi102.stdout:64 writesame 0x19c60b1 thru 0x19cfbee data_size 0x1f(0x9b3e bytes) 2024-09-16T11:08:20.040 INFO:teuthology.orchestra.run.smithi102.stdout:65 writesame 0x49e2d32 thru 0x49e89f9 data_size 0x4(0x5cc8 bytes) 2024-09-16T11:08:20.058 INFO:teuthology.orchestra.run.smithi102.stdout:67 writesame 0x503fdf3 thru 0x504f816 data_size 0xe(0xfa24 bytes) 2024-09-16T11:08:20.059 INFO:teuthology.orchestra.run.smithi102.stdout:68 read 0x2b5648 thru 0x2bb72e (0x60e7 bytes) 2024-09-16T11:08:20.060 INFO:teuthology.orchestra.run.smithi102.stdout:69 write 0xaa0310 thru 0xaaecd4 (0xe9c5 bytes) 2024-09-16T11:08:20.066 INFO:teuthology.orchestra.run.smithi102.stdout:71 read 0x35beb16 thru 0x35c044f (0x193a bytes) 2024-09-16T11:08:20.066 INFO:teuthology.orchestra.run.smithi102.stdout:72 write 0x1890908 thru 0x189488c (0x3f85 bytes) 2024-09-16T11:08:20.069 INFO:teuthology.orchestra.run.smithi102.stdout:truncating to largest ever: 0x5d89609 2024-09-16T11:08:20.075 INFO:teuthology.orchestra.run.smithi102.stdout:73 trunc from 0x504f817 to 0x5d89609 2024-09-16T11:08:20.108 INFO:teuthology.orchestra.run.smithi102.stdout:74 write 0x6aa09da thru 0x6aa1421 (0xa48 bytes) 2024-09-16T11:08:20.113 INFO:teuthology.orchestra.run.smithi102.stdout:75 punch from 0x42058da to 0x420e98f, (0x90b5 bytes) 2024-09-16T11:08:20.115 INFO:teuthology.orchestra.run.smithi102.stdout:76 writesame 0x41a15ba thru 0x41afc49 data_size 0x8(0xe690 bytes) 2024-09-16T11:08:20.123 INFO:teuthology.orchestra.run.smithi102.stdout:78 write 0x21c304e thru 0x21c683d (0x37f0 bytes) 2024-09-16T11:08:20.157 INFO:teuthology.orchestra.run.smithi102.stdout:80 trunc from 0x6aa1422 to 0x2e71eed 2024-09-16T11:08:20.190 INFO:teuthology.orchestra.run.smithi102.stdout:81 writesame 0x1f9ff75 thru 0x1facb8d data_size 0x1(0xcc19 bytes) 2024-09-16T11:08:20.222 INFO:teuthology.orchestra.run.smithi102.stdout:truncating to largest ever: 0x787570f 2024-09-16T11:08:20.253 INFO:teuthology.orchestra.run.smithi102.stdout:82 trunc from 0x2e71eed to 0x787570f 2024-09-16T11:08:20.268 INFO:teuthology.orchestra.run.smithi102.stdout:83 read 0x3236121 thru 0x323b8ce (0x57ae bytes) 2024-09-16T11:08:20.268 INFO:teuthology.orchestra.run.smithi102.stdout:85 read 0x476c771 thru 0x4779a25 (0xd2b5 bytes) 2024-09-16T11:08:20.270 INFO:teuthology.orchestra.run.smithi102.stdout:86 write 0x51e37d4 thru 0x51e91ad (0x59da bytes) 2024-09-16T11:08:20.276 INFO:teuthology.orchestra.run.smithi102.stdout:87 write 0x764c7ef thru 0x764f7e2 (0x2ff4 bytes) 2024-09-16T11:08:20.280 INFO:teuthology.orchestra.run.smithi102.stdout:compare and write disabled 2024-09-16T11:08:20.281 INFO:teuthology.orchestra.run.smithi102.stdout:90 write 0x4593805 thru 0x45997b2 (0x5fae bytes) 2024-09-16T11:08:20.286 INFO:teuthology.orchestra.run.smithi102.stdout:92 write 0x5e0e01c thru 0x5e0ec84 (0xc69 bytes) 2024-09-16T11:08:20.329 INFO:teuthology.orchestra.run.smithi102.stdout:95 trunc from 0x787570f to 0x1dcf613 2024-09-16T11:08:20.358 INFO:teuthology.orchestra.run.smithi102.stdout:96 read 0xd02d0a thru 0xd0c10c (0x9403 bytes) 2024-09-16T11:08:20.402 INFO:teuthology.orchestra.run.smithi102.stdout:97 write 0x608e01d thru 0x6094e42 (0x6e26 bytes) 2024-09-16T11:08:20.407 INFO:teuthology.orchestra.run.smithi102.stdout:99 punch from 0x507d450 to 0x507f200, (0x1db0 bytes) 2024-09-16T11:08:20.408 INFO:teuthology.orchestra.run.smithi102.stdout:100 writesame 0x4058319 thru 0x40635d7 data_size 0x7(0xb2bf bytes) 2024-09-16T11:08:20.415 INFO:teuthology.orchestra.run.smithi102.stdout:101 read 0xc9712 thru 0xd075c (0x704b bytes) 2024-09-16T11:08:20.417 INFO:teuthology.orchestra.run.smithi102.stdout:103 writesame 0x243b0b thru 0x24b96c data_size 0x7(0x7e62 bytes) 2024-09-16T11:08:20.428 INFO:teuthology.orchestra.run.smithi102.stdout:104 trunc from 0x6094e43 to 0x55cd7fa 2024-09-16T11:08:20.451 INFO:teuthology.orchestra.run.smithi102.stdout:106 write 0xa6d5f3 thru 0xa7cffb (0xfa09 bytes) 2024-09-16T11:08:20.470 INFO:teuthology.orchestra.run.smithi102.stdout:107 write 0x5af3187 thru 0x5af8134 (0x4fae bytes) 2024-09-16T11:08:20.483 INFO:teuthology.orchestra.run.smithi102.stdout:108 trunc from 0x5af8135 to 0x6dde713 2024-09-16T11:08:20.498 INFO:teuthology.orchestra.run.smithi102.stdout:109 writesame 0x18fb9bf thru 0x18ff141 data_size 0x3(0x3783 bytes) 2024-09-16T11:08:20.502 INFO:teuthology.orchestra.run.smithi102.stdout:111 write 0x208d5d9 thru 0x209414d (0x6b75 bytes) 2024-09-16T11:08:20.509 INFO:teuthology.orchestra.run.smithi102.stdout:113 write 0x39fdb37 thru 0x3a0b1f3 (0xd6bd bytes) 2024-09-16T11:08:20.515 INFO:teuthology.orchestra.run.smithi102.stdout:115 write 0x3d267e0 thru 0x3d2afa1 (0x47c2 bytes) 2024-09-16T11:08:20.519 INFO:teuthology.orchestra.run.smithi102.stdout:116 read 0x5f237cb thru 0x5f2949a (0x5cd0 bytes) 2024-09-16T11:08:20.524 INFO:teuthology.orchestra.run.smithi102.stdout:117 trunc from 0x6dde713 to 0x775eba8 2024-09-16T11:08:20.538 INFO:teuthology.orchestra.run.smithi102.stdout:118 writesame 0x323fcf9 thru 0x32447d0 data_size 0x14(0x4ad8 bytes) 2024-09-16T11:08:20.559 INFO:teuthology.orchestra.run.smithi102.stdout:119 writesame 0x7c336bf thru 0x7c393f1 data_size 0x21(0x5d33 bytes) 2024-09-16T11:08:20.559 INFO:teuthology.orchestra.run.smithi102.stdout:120 read 0x6f3f4bf thru 0x6f4645e (0x6fa0 bytes) 2024-09-16T11:08:20.560 INFO:teuthology.orchestra.run.smithi102.stdout:121 writesame 0x56c92e6 thru 0x56d306d data_size 0x47(0x9d88 bytes) 2024-09-16T11:08:20.565 INFO:teuthology.orchestra.run.smithi102.stdout:122 punch from 0x1bad610 to 0x1bb1332, (0x3d22 bytes) 2024-09-16T11:08:20.565 INFO:teuthology.orchestra.run.smithi102.stdout:123 read 0x290f15e thru 0x291d50e (0xe3b1 bytes) 2024-09-16T11:08:20.566 INFO:teuthology.orchestra.run.smithi102.stdout:124 read 0x683849 thru 0x691445 (0xdbfd bytes) 2024-09-16T11:08:20.614 INFO:teuthology.orchestra.run.smithi102.stdout:125 trunc from 0x7c393f2 to 0xd69c1a 2024-09-16T11:08:20.706 INFO:teuthology.orchestra.run.smithi102.stdout:126 write 0x6eb375d thru 0x6ebca3e (0x92e2 bytes) 2024-09-16T11:08:20.711 INFO:teuthology.orchestra.run.smithi102.stdout:127 punch from 0x25fc335 to 0x260b108, (0xedd3 bytes) 2024-09-16T11:08:20.712 INFO:teuthology.orchestra.run.smithi102.stdout:129 write 0x4310945 thru 0x43194e7 (0x8ba3 bytes) 2024-09-16T11:08:20.717 INFO:teuthology.orchestra.run.smithi102.stdout:130 read 0x1ec1a19 thru 0x1ec4c6d (0x3255 bytes) 2024-09-16T11:08:20.742 INFO:teuthology.orchestra.run.smithi102.stdout:131 writesame 0x75ce3e2 thru 0x75de0c6 data_size 0x1(0xfce5 bytes) 2024-09-16T11:08:20.743 INFO:teuthology.orchestra.run.smithi102.stdout:132 punch from 0x54305fe to 0x543a9b2, (0xa3b4 bytes) 2024-09-16T11:08:20.743 INFO:teuthology.orchestra.run.smithi102.stdout:133 write 0x196502e thru 0x1966b22 (0x1af5 bytes) 2024-09-16T11:08:20.747 INFO:teuthology.orchestra.run.smithi102.stdout:134 read 0x561fbe8 thru 0x562889b (0x8cb4 bytes) 2024-09-16T11:08:20.749 INFO:teuthology.orchestra.run.smithi102.stdout:135 writesame 0x2d4ea63 thru 0x2d5571f data_size 0x9(0x6cbd bytes) 2024-09-16T11:08:20.754 INFO:teuthology.orchestra.run.smithi102.stdout:136 read 0x522ad12 thru 0x5239d66 (0xf055 bytes) 2024-09-16T11:08:20.766 INFO:teuthology.orchestra.run.smithi102.stdout:137 trunc from 0x75de0c7 to 0x5dd3ce0 2024-09-16T11:08:20.787 INFO:teuthology.orchestra.run.smithi102.stdout:138 writesame 0x5c3036 thru 0x5c8535 data_size 0x80(0x5500 bytes) 2024-09-16T11:08:20.792 INFO:teuthology.orchestra.run.smithi102.stdout:139 read 0x41171c0 thru 0x41175c5 (0x406 bytes) 2024-09-16T11:08:20.818 INFO:teuthology.orchestra.run.smithi102.stdout:143 writesame 0x5e8d5df thru 0x5e91d9b data_size 0x1(0x47bd bytes) 2024-09-16T11:08:20.822 INFO:teuthology.orchestra.run.smithi102.stdout:145 trunc from 0x5e91d9c to 0x56742f2 2024-09-16T11:08:20.838 INFO:teuthology.orchestra.run.smithi102.stdout:146 writesame 0x11a7906 thru 0x11acac7 data_size 0x73(0x51c2 bytes) 2024-09-16T11:08:20.863 INFO:teuthology.orchestra.run.smithi102.stdout:148 trunc from 0x56742f2 to 0x2758c61 2024-09-16T11:08:20.888 INFO:teuthology.orchestra.run.smithi102.stdout:149 punch from 0x3e9776 to 0x3ef95c, (0x61e6 bytes) 2024-09-16T11:08:20.921 INFO:teuthology.orchestra.run.smithi102.stdout:151 write 0x4e908ba thru 0x4e9f2bd (0xea04 bytes) 2024-09-16T11:08:20.926 INFO:teuthology.orchestra.run.smithi102.stdout:compare and write disabled 2024-09-16T11:08:20.965 INFO:teuthology.orchestra.run.smithi102.stdout:154 writesame 0x7a529b7 thru 0x7a5dd9c data_size 0x1(0xb3e6 bytes) 2024-09-16T11:08:20.965 INFO:teuthology.orchestra.run.smithi102.stdout:155 read 0x17fa47b thru 0x180683b (0xc3c1 bytes) 2024-09-16T11:08:20.970 INFO:teuthology.orchestra.run.smithi102.stdout:157 writesame 0x51386e5 thru 0x5139b88 data_size 0x2(0x14a4 bytes) 2024-09-16T11:08:20.975 INFO:teuthology.orchestra.run.smithi102.stdout:158 write 0x26d660c thru 0x26e177e (0xb173 bytes) 2024-09-16T11:08:20.981 INFO:teuthology.orchestra.run.smithi102.stdout:159 read 0x1e8f88c thru 0x1e9cf43 (0xd6b8 bytes) 2024-09-16T11:08:20.982 INFO:teuthology.orchestra.run.smithi102.stdout:160 write 0xfeba6 thru 0x10a442 (0xb89d bytes) 2024-09-16T11:08:20.986 INFO:teuthology.orchestra.run.smithi102.stdout:161 write 0xeb2a0f thru 0xeb8c1f (0x6211 bytes) 2024-09-16T11:08:20.991 INFO:teuthology.orchestra.run.smithi102.stdout:162 write 0x3f9b558 thru 0x3fa3ede (0x8987 bytes) 2024-09-16T11:08:20.996 INFO:teuthology.orchestra.run.smithi102.stdout:163 read 0x3c18553 thru 0x3c25368 (0xce16 bytes) 2024-09-16T11:08:21.000 INFO:teuthology.orchestra.run.smithi102.stdout:165 writesame 0x53fdd01 thru 0x54095b9 data_size 0xb(0xb8b9 bytes) 2024-09-16T11:08:21.067 INFO:teuthology.orchestra.run.smithi102.stdout:166 trunc from 0x7a5dd9d to 0x5a328d7 2024-09-16T11:08:21.084 INFO:teuthology.orchestra.run.smithi102.stdout:167 read 0x2408c12 thru 0x2413584 (0xa973 bytes) 2024-09-16T11:08:21.086 INFO:teuthology.orchestra.run.smithi102.stdout:168 read 0x2aec4fc thru 0x2af4fb6 (0x8abb bytes) 2024-09-16T11:08:21.087 INFO:teuthology.orchestra.run.smithi102.stdout:169 read 0x39c8eed thru 0x39cc3c7 (0x34db bytes) 2024-09-16T11:08:21.087 INFO:teuthology.orchestra.run.smithi102.stdout:170 read 0x2c7d458 thru 0x2c8a4d8 (0xd081 bytes) 2024-09-16T11:08:21.089 INFO:teuthology.orchestra.run.smithi102.stdout:171 write 0x421e5a7 thru 0x4221af0 (0x354a bytes) 2024-09-16T11:08:21.094 INFO:teuthology.orchestra.run.smithi102.stdout:172 read 0x1f31480 thru 0x1f33d5e (0x28df bytes) 2024-09-16T11:08:21.112 INFO:teuthology.orchestra.run.smithi102.stdout:174 trunc from 0x5a328d7 to 0x3195d97 2024-09-16T11:08:21.143 INFO:teuthology.orchestra.run.smithi102.stdout:176 write 0xa6062c thru 0xa629ae (0x2383 bytes) 2024-09-16T11:08:21.148 INFO:teuthology.orchestra.run.smithi102.stdout:compare and write disabled 2024-09-16T11:08:21.148 INFO:teuthology.orchestra.run.smithi102.stdout:180 read 0x79300d thru 0x79651f (0x3513 bytes) 2024-09-16T11:08:21.156 INFO:teuthology.orchestra.run.smithi102.stdout:181 trunc from 0x3195d97 to 0x3ef0233 2024-09-16T11:08:21.170 INFO:teuthology.orchestra.run.smithi102.stdout:182 write 0x5a9f29 thru 0x5b7cca (0xdda2 bytes) 2024-09-16T11:08:21.174 INFO:teuthology.orchestra.run.smithi102.stdout:compare and write disabled 2024-09-16T11:08:21.174 INFO:teuthology.orchestra.run.smithi102.stdout:184 read 0x9a1b48 thru 0x9af325 (0xd7de bytes) 2024-09-16T11:08:21.176 INFO:teuthology.orchestra.run.smithi102.stdout:185 punch from 0x2139d31 to 0x213d923, (0x3bf2 bytes) 2024-09-16T11:08:21.199 INFO:teuthology.orchestra.run.smithi102.stdout:187 writesame 0x4f7b336 thru 0x4f82658 data_size 0x4b(0x7323 bytes) 2024-09-16T11:08:21.202 INFO:teuthology.orchestra.run.smithi102.stdout:189 writesame 0xc52f43 thru 0xc5963f data_size 0x1(0x66fd bytes) 2024-09-16T11:08:21.244 INFO:teuthology.orchestra.run.smithi102.stdout:190 write 0x6c5ab5c thru 0x6c5ccc4 (0x2169 bytes) 2024-09-16T11:08:21.249 INFO:teuthology.orchestra.run.smithi102.stdout:191 writesame 0x5dde5d5 thru 0x5decc08 data_size 0x12(0xe634 bytes) 2024-09-16T11:08:21.255 INFO:teuthology.orchestra.run.smithi102.stdout:192 read 0x92060c thru 0x92c087 (0xba7c bytes) 2024-09-16T11:08:21.257 INFO:teuthology.orchestra.run.smithi102.stdout:193 write 0x323b2fc thru 0x323bd41 (0xa46 bytes) 2024-09-16T11:08:21.261 INFO:teuthology.orchestra.run.smithi102.stdout:195 read 0x62c8636 thru 0x62d660b (0xdfd6 bytes) 2024-09-16T11:08:21.262 INFO:teuthology.orchestra.run.smithi102.stdout:truncating to largest ever: 0x78e9e28 2024-09-16T11:08:21.268 INFO:teuthology.orchestra.run.smithi102.stdout:196 trunc from 0x6c5ccc5 to 0x78e9e28 2024-09-16T11:08:21.302 INFO:teuthology.orchestra.run.smithi102.stdout:197 trunc from 0x78e9e28 to 0x518fede 2024-09-16T11:08:21.323 INFO:teuthology.orchestra.run.smithi102.stdout:truncating to largest ever: 0x7c893a3 2024-09-16T11:08:21.341 INFO:teuthology.orchestra.run.smithi102.stdout:201 trunc from 0x518fede to 0x7c893a3 2024-09-16T11:08:21.356 INFO:teuthology.orchestra.run.smithi102.stdout:compare and write disabled 2024-09-16T11:08:21.356 INFO:teuthology.orchestra.run.smithi102.stdout:204 punch from 0x57b6c7a to 0x57c1f42, (0xb2c8 bytes) 2024-09-16T11:08:21.357 INFO:teuthology.orchestra.run.smithi102.stdout:205 write 0x1ede3a3 thru 0x1eebd30 (0xd98e bytes) 2024-09-16T11:08:21.362 INFO:teuthology.orchestra.run.smithi102.stdout:206 write 0x37746a7 thru 0x377694b (0x22a5 bytes) 2024-09-16T11:08:21.368 INFO:teuthology.orchestra.run.smithi102.stdout:208 clone 1 order 21 su 65536 sc 12 2024-09-16T11:08:22.662 INFO:teuthology.orchestra.run.smithi102.stdout:209 trunc from 0x7c893a3 to 0xf89bb2 2024-09-16T11:08:23.032 INFO:teuthology.orchestra.run.smithi102.stdout:210 writesame 0x5bb5935 thru 0x5bba68d data_size 0x1(0x4d59 bytes) 2024-09-16T11:08:23.033 INFO:teuthology.orchestra.run.smithi102.stdout:compare and write disabled 2024-09-16T11:08:23.060 INFO:teuthology.orchestra.run.smithi102.stdout:213 write 0x77ff7ce thru 0x780bd4e (0xc581 bytes) 2024-09-16T11:08:23.073 INFO:teuthology.orchestra.run.smithi102.stdout:214 write 0x46ca607 thru 0x46d72fd (0xccf7 bytes) 2024-09-16T11:08:23.082 INFO:teuthology.orchestra.run.smithi102.stdout:215 writesame 0x75435d1 thru 0x755153b data_size 0x9b(0xdf6b bytes) 2024-09-16T11:08:23.093 INFO:teuthology.orchestra.run.smithi102.stdout:216 read 0x65a75d2 thru 0x65b0b2b (0x955a bytes) 2024-09-16T11:08:23.094 INFO:teuthology.orchestra.run.smithi102.stdout:217 write 0x4871ce5 thru 0x48755ec (0x3908 bytes) 2024-09-16T11:08:23.099 INFO:teuthology.orchestra.run.smithi102.stdout:218 write 0x179d4cc thru 0x17a6b94 (0x96c9 bytes) 2024-09-16T11:08:23.105 INFO:teuthology.orchestra.run.smithi102.stdout:219 punch from 0x2aa68ea to 0x2ab14a8, (0xabbe bytes) 2024-09-16T11:08:23.106 INFO:teuthology.orchestra.run.smithi102.stdout:compare and write disabled 2024-09-16T11:08:23.106 INFO:teuthology.orchestra.run.smithi102.stdout:221 write 0x657f97c thru 0x6587f2e (0x85b3 bytes) 2024-09-16T11:08:23.112 INFO:teuthology.orchestra.run.smithi102.stdout:222 write 0x6c4a6ed thru 0x6c4e610 (0x3f24 bytes) 2024-09-16T11:08:23.115 INFO:teuthology.orchestra.run.smithi102.stdout:compare and write disabled 2024-09-16T11:08:23.115 INFO:teuthology.orchestra.run.smithi102.stdout:truncating to largest ever: 0x7f93b2a 2024-09-16T11:08:23.118 INFO:teuthology.orchestra.run.smithi102.stdout:225 trunc from 0x780bd4f to 0x7f93b2a 2024-09-16T11:08:23.132 INFO:teuthology.orchestra.run.smithi102.stdout:227 write 0xe0be3 thru 0xe3472 (0x2890 bytes) 2024-09-16T11:08:23.134 INFO:teuthology.orchestra.run.smithi102.stdout:228 punch from 0x364063c to 0x3647b11, (0x74d5 bytes) 2024-09-16T11:08:23.135 INFO:teuthology.orchestra.run.smithi102.stdout:229 read 0x4426111 thru 0x4434b71 (0xea61 bytes) 2024-09-16T11:08:23.137 INFO:teuthology.orchestra.run.smithi102.stdout:230 write 0x42bb1ff thru 0x42c8406 (0xd208 bytes) 2024-09-16T11:08:23.144 INFO:teuthology.orchestra.run.smithi102.stdout:231 read 0x5830564 thru 0x583e895 (0xe332 bytes) 2024-09-16T11:08:23.146 INFO:teuthology.orchestra.run.smithi102.stdout:236 writesame 0x56c9bc8 thru 0x56cb3d9 data_size 0x3(0x1812 bytes) 2024-09-16T11:08:23.151 INFO:teuthology.orchestra.run.smithi102.stdout:compare and write disabled 2024-09-16T11:08:23.151 INFO:teuthology.orchestra.run.smithi102.stdout:239 read 0x365239e thru 0x3653f01 (0x1b64 bytes) 2024-09-16T11:08:23.153 INFO:teuthology.orchestra.run.smithi102.stdout:240 read 0x349c64b thru 0x349eddd (0x2793 bytes) 2024-09-16T11:08:23.153 INFO:teuthology.orchestra.run.smithi102.stdout:243 read 0x220712d thru 0x2211c64 (0xab38 bytes) 2024-09-16T11:08:23.155 INFO:teuthology.orchestra.run.smithi102.stdout:244 write 0x4b0c2a0 thru 0x4b19616 (0xd377 bytes) 2024-09-16T11:08:23.162 INFO:teuthology.orchestra.run.smithi102.stdout:246 writesame 0x409622 thru 0x410377 data_size 0x2d(0x6d56 bytes) 2024-09-16T11:08:23.173 INFO:teuthology.orchestra.run.smithi102.stdout:247 write 0x47d3ba7 thru 0x47dd401 (0x985b bytes) 2024-09-16T11:08:23.232 INFO:teuthology.orchestra.run.smithi102.stdout:248 trunc from 0x7f93b2a to 0x1e7d3a 2024-09-16T11:08:23.276 INFO:teuthology.orchestra.run.smithi102.stdout:249 write 0x429b5 thru 0x429c8 (0x14 bytes) 2024-09-16T11:08:23.309 INFO:teuthology.orchestra.run.smithi102.stdout:251 writesame 0x1ea7cc5 thru 0x1eafb84 data_size 0x1a(0x7ec0 bytes) 2024-09-16T11:08:23.316 INFO:teuthology.orchestra.run.smithi102.stdout:253 trunc from 0x1eafb85 to 0x3011f90 2024-09-16T11:08:23.331 INFO:teuthology.orchestra.run.smithi102.stdout:254 write 0x22c2fc1 thru 0x22c5c94 (0x2cd4 bytes) 2024-09-16T11:08:23.336 INFO:teuthology.orchestra.run.smithi102.stdout:255 read 0x2f4fdd8 thru 0x2f5c61a (0xc843 bytes) 2024-09-16T11:08:23.337 INFO:teuthology.orchestra.run.smithi102.stdout:257 write 0x7ee655 thru 0x7efa57 (0x1403 bytes) 2024-09-16T11:08:23.359 INFO:teuthology.orchestra.run.smithi102.stdout:258 write 0x3c45bb0 thru 0x3c45bf6 (0x47 bytes) 2024-09-16T11:08:23.363 INFO:teuthology.orchestra.run.smithi102.stdout:259 read 0x1c07799 thru 0x1c131f3 (0xba5b bytes) 2024-09-16T11:08:23.367 INFO:teuthology.orchestra.run.smithi102.stdout:261 write 0x41da8e thru 0x427b55 (0xa0c8 bytes) 2024-09-16T11:08:23.373 INFO:teuthology.orchestra.run.smithi102.stdout:262 trunc from 0x3c45bf7 to 0x41f7f79 2024-09-16T11:08:23.388 INFO:teuthology.orchestra.run.smithi102.stdout:compare and write disabled 2024-09-16T11:08:23.405 INFO:teuthology.orchestra.run.smithi102.stdout:264 write 0x478e22e thru 0x4794b5d (0x6930 bytes) 2024-09-16T11:08:23.417 INFO:teuthology.orchestra.run.smithi102.stdout:265 trunc from 0x4794b5e to 0x517add4 2024-09-16T11:08:23.431 INFO:teuthology.orchestra.run.smithi102.stdout:compare and write disabled 2024-09-16T11:08:23.431 INFO:teuthology.orchestra.run.smithi102.stdout:267 write 0x4d74b11 thru 0x4d80231 (0xb721 bytes) 2024-09-16T11:08:23.442 INFO:teuthology.orchestra.run.smithi102.stdout:270 read 0x3980392 thru 0x3986b4a (0x67b9 bytes) 2024-09-16T11:08:23.443 INFO:teuthology.orchestra.run.smithi102.stdout:271 read 0x4286aca thru 0x42925fd (0xbb34 bytes) 2024-09-16T11:08:23.450 INFO:teuthology.orchestra.run.smithi102.stdout:272 trunc from 0x517add4 to 0x5e2b4ac 2024-09-16T11:08:23.464 INFO:teuthology.orchestra.run.smithi102.stdout:273 read 0xeaf42f thru 0xeb584c (0x641e bytes) 2024-09-16T11:08:23.468 INFO:teuthology.orchestra.run.smithi102.stdout:274 writesame 0x4419ac4 thru 0x442505d data_size 0x5(0xb59a bytes) 2024-09-16T11:08:23.573 INFO:teuthology.orchestra.run.smithi102.stdout:275 read 0x1c0d709 thru 0x1c16bee (0x94e6 bytes) 2024-09-16T11:08:23.575 INFO:teuthology.orchestra.run.smithi102.stdout:compare and write disabled 2024-09-16T11:08:23.575 INFO:teuthology.orchestra.run.smithi102.stdout:278 punch from 0x40b88cd to 0x40bcacf, (0x4202 bytes) 2024-09-16T11:08:23.575 INFO:teuthology.orchestra.run.smithi102.stdout:279 read 0x26ccd36 thru 0x26d855d (0xb828 bytes) 2024-09-16T11:08:23.576 INFO:teuthology.orchestra.run.smithi102.stdout:280 punch from 0x1973248 to 0x197b002, (0x7dba bytes) 2024-09-16T11:08:23.577 INFO:teuthology.orchestra.run.smithi102.stdout:281 clone 2 order 22 su 32768 sc 5 2024-09-16T11:08:24.130 INFO:teuthology.orchestra.run.smithi102.stdout:truncating image image_client.0-clone1 from 0x5e2b4ac (overlap 0x1e7d3a) to 0x124b22 2024-09-16T11:08:24.418 INFO:teuthology.orchestra.run.smithi102.stdout:checking clone #0, image image_client.0 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent1 2024-09-16T11:08:26.622 INFO:teuthology.orchestra.run.smithi102.stdout:283 punch from 0x4eb10dd to 0x4eb9225, (0x8148 bytes) 2024-09-16T11:08:26.636 INFO:teuthology.orchestra.run.smithi102.stdout:284 trunc from 0x5e2b4ac to 0x616547e 2024-09-16T11:08:26.651 INFO:teuthology.orchestra.run.smithi102.stdout:compare and write disabled 2024-09-16T11:08:26.657 INFO:teuthology.orchestra.run.smithi102.stdout:287 writesame 0x1a01342 thru 0x1a0d9c7 data_size 0x1(0xc686 bytes) 2024-09-16T11:08:26.730 INFO:teuthology.orchestra.run.smithi102.stdout:289 read 0x5cc3c6d thru 0x5cd1877 (0xdc0b bytes) 2024-09-16T11:08:26.735 INFO:teuthology.orchestra.run.smithi102.stdout:compare and write disabled 2024-09-16T11:08:26.737 INFO:teuthology.orchestra.run.smithi102.stdout:292 writesame 0x57735b1 thru 0x5776516 data_size 0x1(0x2f66 bytes) 2024-09-16T11:08:26.769 INFO:teuthology.orchestra.run.smithi102.stdout:293 trunc from 0x616547e to 0x49820a4 2024-09-16T11:08:27.000 INFO:teuthology.orchestra.run.smithi102.stdout:compare and write disabled 2024-09-16T11:08:27.000 INFO:teuthology.orchestra.run.smithi102.stdout:296 punch from 0x1f0eee2 to 0x1f18781, (0x989f bytes) 2024-09-16T11:08:27.000 INFO:teuthology.orchestra.run.smithi102.stdout:297 read 0xbfb033 thru 0xbfe2e6 (0x32b4 bytes) 2024-09-16T11:08:27.002 INFO:teuthology.orchestra.run.smithi102.stdout:298 punch from 0x23b1bb4 to 0x23b56ee, (0x3b3a bytes) 2024-09-16T11:08:27.024 INFO:teuthology.orchestra.run.smithi102.stdout:299 write 0x5a9407a thru 0x5a947bc (0x743 bytes) 2024-09-16T11:08:27.029 INFO:teuthology.orchestra.run.smithi102.stdout:compare and write disabled 2024-09-16T11:08:27.029 INFO:teuthology.orchestra.run.smithi102.stdout:303 punch from 0x88636d to 0x889fb5, (0x3c48 bytes) 2024-09-16T11:08:27.030 INFO:teuthology.orchestra.run.smithi102.stdout:compare and write disabled 2024-09-16T11:08:27.030 INFO:teuthology.orchestra.run.smithi102.stdout:compare and write disabled 2024-09-16T11:08:27.054 INFO:teuthology.orchestra.run.smithi102.stdout:309 write 0x6dd9211 thru 0x6de7045 (0xde35 bytes) 2024-09-16T11:08:27.062 INFO:teuthology.orchestra.run.smithi102.stdout:310 read 0x6539a2f thru 0x6541453 (0x7a25 bytes) 2024-09-16T11:08:27.107 INFO:teuthology.orchestra.run.smithi102.stdout:311 trunc from 0x6de7046 to 0xa84bd3 2024-09-16T11:08:27.526 INFO:teuthology.orchestra.run.smithi102.stdout:312 trunc from 0xa84bd3 to 0x80f723 2024-09-16T11:08:27.599 INFO:teuthology.orchestra.run.smithi102.stdout:313 writesame 0x574f95c thru 0x5757ab2 data_size 0xd(0x8157 bytes) 2024-09-16T11:08:27.599 INFO:teuthology.orchestra.run.smithi102.stdout:314 write 0x78cd15 thru 0x7910cc (0x43b8 bytes) 2024-09-16T11:08:27.604 INFO:teuthology.orchestra.run.smithi102.stdout:compare and write disabled 2024-09-16T11:08:27.605 INFO:teuthology.orchestra.run.smithi102.stdout:317 writesame 0x6a290a thru 0x6ad007 data_size 0x1e(0xa6fe bytes) 2024-09-16T11:08:27.624 INFO:teuthology.orchestra.run.smithi102.stdout:319 read 0x1de3c97 thru 0x1dee33c (0xa6a6 bytes) 2024-09-16T11:08:27.637 INFO:teuthology.orchestra.run.smithi102.stdout:320 trunc from 0x5757ab3 to 0x3c2a2ff 2024-09-16T11:08:27.654 INFO:teuthology.orchestra.run.smithi102.stdout:321 punch from 0x2225478 to 0x2226fcc, (0x1b54 bytes) 2024-09-16T11:08:27.663 INFO:teuthology.orchestra.run.smithi102.stdout:322 trunc from 0x3c2a2ff to 0x50b2563 2024-09-16T11:08:27.678 INFO:teuthology.orchestra.run.smithi102.stdout:323 read 0x2026a1b thru 0x2035334 (0xe91a bytes) 2024-09-16T11:08:27.680 INFO:teuthology.orchestra.run.smithi102.stdout:324 punch from 0x2e35883 to 0x2e3df11, (0x868e bytes) 2024-09-16T11:08:27.680 INFO:teuthology.orchestra.run.smithi102.stdout:325 punch from 0x313c958 to 0x31486f9, (0xbda1 bytes) 2024-09-16T11:08:27.680 INFO:teuthology.orchestra.run.smithi102.stdout:326 writesame 0x2b1c797 thru 0x2b1ca39 data_size 0x9(0x2a3 bytes) 2024-09-16T11:08:27.711 INFO:teuthology.orchestra.run.smithi102.stdout:327 write 0x68dca85 thru 0x68e09ab (0x3f27 bytes) 2024-09-16T11:08:27.732 INFO:teuthology.orchestra.run.smithi102.stdout:328 trunc from 0x68e09ac to 0x470eda4 2024-09-16T11:08:27.755 INFO:teuthology.orchestra.run.smithi102.stdout:compare and write disabled 2024-09-16T11:08:27.788 INFO:teuthology.orchestra.run.smithi102.stdout:332 write 0x6eb88e2 thru 0x6ebaa80 (0x219f bytes) 2024-09-16T11:08:27.793 INFO:teuthology.orchestra.run.smithi102.stdout:333 write 0x24ed0a5 thru 0x24f8fd8 (0xbf34 bytes) 2024-09-16T11:08:27.806 INFO:teuthology.orchestra.run.smithi102.stdout:335 write 0x46465cf thru 0x464935d (0x2d8f bytes) 2024-09-16T11:08:27.821 INFO:teuthology.orchestra.run.smithi102.stdout:336 read 0x5a97e thru 0x68926 (0xdfa9 bytes) 2024-09-16T11:08:27.843 INFO:teuthology.orchestra.run.smithi102.stdout:338 writesame 0x70d0751 thru 0x70d2fe9 data_size 0x1(0x2899 bytes) 2024-09-16T11:08:27.843 INFO:teuthology.orchestra.run.smithi102.stdout:340 punch from 0x2b7df05 to 0x2b8c85d, (0xe958 bytes) 2024-09-16T11:08:27.865 INFO:teuthology.orchestra.run.smithi102.stdout:341 writesame 0x7c8ddd4 thru 0x7c969ef data_size 0x3d(0x8c1c bytes) 2024-09-16T11:08:27.917 INFO:teuthology.orchestra.run.smithi102.stdout:344 trunc from 0x7c969f0 to 0x377942 2024-09-16T11:08:28.002 INFO:teuthology.orchestra.run.smithi102.stdout:346 write 0x4d0f9cd thru 0x4d11c69 (0x229d bytes) 2024-09-16T11:08:28.034 INFO:teuthology.orchestra.run.smithi102.stdout:347 writesame 0x6343bc3 thru 0x635343a data_size 0x4(0xf878 bytes) 2024-09-16T11:08:28.035 INFO:teuthology.orchestra.run.smithi102.stdout:348 read 0x1c06e88 thru 0x1c08d1d (0x1e96 bytes) 2024-09-16T11:08:28.036 INFO:teuthology.orchestra.run.smithi102.stdout:349 punch from 0x6232cb9 to 0x62418ea, (0xec31 bytes) 2024-09-16T11:08:28.052 INFO:teuthology.orchestra.run.smithi102.stdout:350 write 0x66d1735 thru 0x66d601a (0x48e6 bytes) 2024-09-16T11:08:28.058 INFO:teuthology.orchestra.run.smithi102.stdout:352 write 0x40b8ba7 thru 0x40bd02d (0x4487 bytes) 2024-09-16T11:08:28.061 INFO:teuthology.orchestra.run.smithi102.stdout:353 write 0x2753eeb thru 0x275b02f (0x7145 bytes) 2024-09-16T11:08:28.073 INFO:teuthology.orchestra.run.smithi102.stdout:354 writesame 0x18becce thru 0x18c05ac data_size 0x1(0x18df bytes) 2024-09-16T11:08:28.085 INFO:teuthology.orchestra.run.smithi102.stdout:357 read 0x4cdf2ef thru 0x4ced564 (0xe276 bytes) 2024-09-16T11:08:28.089 INFO:teuthology.orchestra.run.smithi102.stdout:359 read 0x636389b thru 0x63648e7 (0x104d bytes) 2024-09-16T11:08:28.089 INFO:teuthology.orchestra.run.smithi102.stdout:360 read 0x32548e6 thru 0x32596d5 (0x4df0 bytes) 2024-09-16T11:08:28.090 INFO:teuthology.orchestra.run.smithi102.stdout:361 punch from 0x1a47cc2 to 0x1a490ef, (0x142d bytes) 2024-09-16T11:08:28.090 INFO:teuthology.orchestra.run.smithi102.stdout:365 write 0x3cb8489 thru 0x3cbcc7e (0x47f6 bytes) 2024-09-16T11:08:28.096 INFO:teuthology.orchestra.run.smithi102.stdout:366 writesame 0x4deb853 thru 0x4df4ca7 data_size 0x17(0x9455 bytes) 2024-09-16T11:08:28.120 INFO:teuthology.orchestra.run.smithi102.stdout:367 write 0x1d621b7 thru 0x1d70eb3 (0xecfd bytes) 2024-09-16T11:08:28.130 INFO:teuthology.orchestra.run.smithi102.stdout:368 write 0x358b30c thru 0x359a650 (0xf345 bytes) 2024-09-16T11:08:28.141 INFO:teuthology.orchestra.run.smithi102.stdout:compare and write disabled 2024-09-16T11:08:28.177 INFO:teuthology.orchestra.run.smithi102.stdout:370 trunc from 0x66d601b to 0x1a57d31 2024-09-16T11:08:28.237 INFO:teuthology.orchestra.run.smithi102.stdout:371 trunc from 0x1a57d31 to 0x5188847 2024-09-16T11:08:28.254 INFO:teuthology.orchestra.run.smithi102.stdout:372 read 0x5595e9 thru 0x56051a (0x6f32 bytes) 2024-09-16T11:08:28.273 INFO:teuthology.orchestra.run.smithi102.stdout:373 trunc from 0x5188847 to 0x2a07c52 2024-09-16T11:08:28.342 INFO:teuthology.orchestra.run.smithi102.stdout:375 write 0x7f8d1d4 thru 0x7f91668 (0x4495 bytes) 2024-09-16T11:08:28.350 INFO:teuthology.orchestra.run.smithi102.stdout:376 write 0x96fffb thru 0x97611f (0x6125 bytes) 2024-09-16T11:08:28.355 INFO:teuthology.orchestra.run.smithi102.stdout:377 punch from 0x5ecf79 to 0x5f372c, (0x67b3 bytes) 2024-09-16T11:08:28.359 INFO:teuthology.orchestra.run.smithi102.stdout:378 writesame 0x1a05655 thru 0x1a13e60 data_size 0x2(0xe80c bytes) 2024-09-16T11:08:28.672 INFO:teuthology.orchestra.run.smithi102.stdout:380 read 0x78a3ea thru 0x794e28 (0xaa3f bytes) 2024-09-16T11:08:28.676 INFO:teuthology.orchestra.run.smithi102.stdout:381 writesame 0x6f4ff43 thru 0x6f5c55a data_size 0xc(0xc618 bytes) 2024-09-16T11:08:28.728 INFO:teuthology.orchestra.run.smithi102.stdout:382 read 0x547d705 thru 0x5486060 (0x895c bytes) 2024-09-16T11:08:28.730 INFO:teuthology.orchestra.run.smithi102.stdout:384 writesame 0x25c38a1 thru 0x25d1cdc data_size 0x36(0xe43c bytes) 2024-09-16T11:08:28.745 INFO:teuthology.orchestra.run.smithi102.stdout:385 read 0x701c0a8 thru 0x7020c00 (0x4b59 bytes) 2024-09-16T11:08:28.747 INFO:teuthology.orchestra.run.smithi102.stdout:388 punch from 0x7670503 to 0x767337e, (0x2e7b bytes) 2024-09-16T11:08:28.748 INFO:teuthology.orchestra.run.smithi102.stdout:389 writesame 0x644359b thru 0x644c196 data_size 0x44(0x8bfc bytes) 2024-09-16T11:08:28.758 INFO:teuthology.orchestra.run.smithi102.stdout:390 writesame 0x2adaabd thru 0x2ae1b6c data_size 0x18(0x70b0 bytes) 2024-09-16T11:08:28.777 INFO:teuthology.orchestra.run.smithi102.stdout:compare and write disabled 2024-09-16T11:08:28.777 INFO:teuthology.orchestra.run.smithi102.stdout:394 read 0x877e71 thru 0x8802c3 (0x8453 bytes) 2024-09-16T11:08:28.780 INFO:teuthology.orchestra.run.smithi102.stdout:compare and write disabled 2024-09-16T11:08:28.781 INFO:teuthology.orchestra.run.smithi102.stdout:396 write 0x477a5f9 thru 0x47828fd (0x8305 bytes) 2024-09-16T11:08:28.789 INFO:teuthology.orchestra.run.smithi102.stdout:compare and write disabled 2024-09-16T11:08:28.789 INFO:teuthology.orchestra.run.smithi102.stdout:400 punch from 0x215b4f9 to 0x215cac8, (0x15cf bytes) 2024-09-16T11:08:28.789 INFO:teuthology.orchestra.run.smithi102.stdout:401 punch from 0x6c24dbc to 0x6c2b1d6, (0x641a bytes) 2024-09-16T11:08:28.796 INFO:teuthology.orchestra.run.smithi102.stdout:403 writesame 0x644e11a thru 0x645cb93 data_size 0x1(0xea7a bytes) 2024-09-16T11:08:28.865 INFO:teuthology.orchestra.run.smithi102.stdout:404 trunc from 0x7f91669 to 0xee2d57 2024-09-16T11:08:28.958 INFO:teuthology.orchestra.run.smithi102.stdout:405 write 0x6c70a57 thru 0x6c7fd15 (0xf2bf bytes) 2024-09-16T11:08:28.966 INFO:teuthology.orchestra.run.smithi102.stdout:406 write 0x20e670c thru 0x20f22b0 (0xbba5 bytes) 2024-09-16T11:08:28.979 INFO:teuthology.orchestra.run.smithi102.stdout:compare and write disabled 2024-09-16T11:08:28.980 INFO:teuthology.orchestra.run.smithi102.stdout:409 punch from 0xcab795 to 0xcad5d8, (0x1e43 bytes) 2024-09-16T11:08:28.980 INFO:teuthology.orchestra.run.smithi102.stdout:410 read 0x62f204f thru 0x62f9921 (0x78d3 bytes) 2024-09-16T11:08:29.000 INFO:teuthology.orchestra.run.smithi102.stdout:411 write 0x76b7f22 thru 0x76b9bee (0x1ccd bytes) 2024-09-16T11:08:29.007 INFO:teuthology.orchestra.run.smithi102.stdout:413 write 0x43658db thru 0x4368963 (0x3089 bytes) 2024-09-16T11:08:29.014 INFO:teuthology.orchestra.run.smithi102.stdout:compare and write disabled 2024-09-16T11:08:29.014 INFO:teuthology.orchestra.run.smithi102.stdout:compare and write disabled 2024-09-16T11:08:29.014 INFO:teuthology.orchestra.run.smithi102.stdout:417 punch from 0x62e8e98 to 0x62ef6b6, (0x681e bytes) 2024-09-16T11:08:29.037 INFO:teuthology.orchestra.run.smithi102.stdout:418 trunc from 0x76b9bef to 0x41a077c 2024-09-16T11:08:29.072 INFO:teuthology.orchestra.run.smithi102.stdout:419 trunc from 0x41a077c to 0x4f0b984 2024-09-16T11:08:29.091 INFO:teuthology.orchestra.run.smithi102.stdout:420 trunc from 0x4f0b984 to 0x55d4f4f 2024-09-16T11:08:29.105 INFO:teuthology.orchestra.run.smithi102.stdout:423 punch from 0x106726 to 0x10a6e5, (0x3fbf bytes) 2024-09-16T11:08:29.124 INFO:teuthology.orchestra.run.smithi102.stdout:424 writesame 0x5f17e93 thru 0x5f23548 data_size 0xe(0xb6b6 bytes) 2024-09-16T11:08:29.124 INFO:teuthology.orchestra.run.smithi102.stdout:425 punch from 0x3c1b958 to 0x3c260b9, (0xa761 bytes) 2024-09-16T11:08:29.149 INFO:teuthology.orchestra.run.smithi102.stdout:426 write 0x749d73d thru 0x749e48a (0xd4e bytes) 2024-09-16T11:08:29.153 INFO:teuthology.orchestra.run.smithi102.stdout:427 read 0x2420355 thru 0x24216e6 (0x1392 bytes) 2024-09-16T11:08:29.154 INFO:teuthology.orchestra.run.smithi102.stdout:429 writesame 0x41c660a thru 0x41c9187 data_size 0x13(0x2b7e bytes) 2024-09-16T11:08:29.168 INFO:teuthology.orchestra.run.smithi102.stdout:430 write 0x72bf29b thru 0x72c953d (0xa2a3 bytes) 2024-09-16T11:08:29.180 INFO:teuthology.orchestra.run.smithi102.stdout:433 read 0x1bf7dda thru 0x1c05b88 (0xddaf bytes) 2024-09-16T11:08:29.182 INFO:teuthology.orchestra.run.smithi102.stdout:435 read 0x3418f56 thru 0x341e1a1 (0x524c bytes) 2024-09-16T11:08:29.182 INFO:teuthology.orchestra.run.smithi102.stdout:436 read 0x4770a87 thru 0x4773c51 (0x31cb bytes) 2024-09-16T11:08:29.184 INFO:teuthology.orchestra.run.smithi102.stdout:compare and write disabled 2024-09-16T11:08:29.219 INFO:teuthology.orchestra.run.smithi102.stdout:438 trunc from 0x749e48b to 0x22e397f 2024-09-16T11:08:29.297 INFO:teuthology.orchestra.run.smithi102.stdout:439 writesame 0x6858575 thru 0x686598a data_size 0x3(0xd416 bytes) 2024-09-16T11:08:29.298 INFO:teuthology.orchestra.run.smithi102.stdout:443 read 0x352069a thru 0x3524ecf (0x4836 bytes) 2024-09-16T11:08:29.325 INFO:teuthology.orchestra.run.smithi102.stdout:444 writesame 0x79fa3fa thru 0x7a03cb4 data_size 0x1(0x98bb bytes) 2024-09-16T11:08:29.326 INFO:teuthology.orchestra.run.smithi102.stdout:446 write 0x71d0c50 thru 0x71d807c (0x742d bytes) 2024-09-16T11:08:29.347 INFO:teuthology.orchestra.run.smithi102.stdout:447 write 0x7a7b2e1 thru 0x7a81046 (0x5d66 bytes) 2024-09-16T11:08:29.393 INFO:teuthology.orchestra.run.smithi102.stdout:448 trunc from 0x7a81047 to 0x1e07c0f 2024-09-16T11:08:29.473 INFO:teuthology.orchestra.run.smithi102.stdout:449 write 0x7ac9f6a thru 0x7ad9533 (0xf5ca bytes) 2024-09-16T11:08:29.482 INFO:teuthology.orchestra.run.smithi102.stdout:450 writesame 0x29cf31 thru 0x2a0470 data_size 0x10(0x3540 bytes) 2024-09-16T11:08:29.494 INFO:teuthology.orchestra.run.smithi102.stdout:451 write 0x4feec9c thru 0x4ff3de9 (0x514e bytes) 2024-09-16T11:08:29.500 INFO:teuthology.orchestra.run.smithi102.stdout:452 read 0x3fedeb6 thru 0x3ff1560 (0x36ab bytes) 2024-09-16T11:08:29.502 INFO:teuthology.orchestra.run.smithi102.stdout:453 write 0x3bbfa5e thru 0x3bc9f7c (0xa51f bytes) 2024-09-16T11:08:29.511 INFO:teuthology.orchestra.run.smithi102.stdout:454 read 0x2d36240 thru 0x2d3e8bb (0x867c bytes) 2024-09-16T11:08:29.513 INFO:teuthology.orchestra.run.smithi102.stdout:457 punch from 0x75d0ab6 to 0x75d7835, (0x6d7f bytes) 2024-09-16T11:08:29.514 INFO:teuthology.orchestra.run.smithi102.stdout:458 punch from 0x476e9b9 to 0x476f6d5, (0xd1c bytes) 2024-09-16T11:08:29.517 INFO:teuthology.orchestra.run.smithi102.stdout:460 trunc from 0x7ad9534 to 0x742fec9 2024-09-16T11:08:29.540 INFO:teuthology.orchestra.run.smithi102.stdout:compare and write disabled 2024-09-16T11:08:29.540 INFO:teuthology.orchestra.run.smithi102.stdout:compare and write disabled 2024-09-16T11:08:29.541 INFO:teuthology.orchestra.run.smithi102.stdout:465 clone 3 order 22 su 262144 sc 9 2024-09-16T11:08:30.175 INFO:teuthology.orchestra.run.smithi102.stdout:truncating image image_client.0-clone2 from 0x742fec9 (overlap 0x377942) to 0x31ed21 2024-09-16T11:08:30.569 INFO:teuthology.orchestra.run.smithi102.stdout:checking clone #1, image image_client.0-clone1 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent2 2024-09-16T11:08:30.613 INFO:teuthology.orchestra.run.smithi102.stdout:466 read 0x5623f2d thru 0x56317ff (0xd8d3 bytes) 2024-09-16T11:08:30.615 INFO:teuthology.orchestra.run.smithi102.stdout:467 read 0x49d068b thru 0x49d6096 (0x5a0c bytes) 2024-09-16T11:08:30.619 INFO:teuthology.orchestra.run.smithi102.stdout:468 write 0x6e60aaf thru 0x6e6a079 (0x95cb bytes) 2024-09-16T11:08:30.636 INFO:teuthology.orchestra.run.smithi102.stdout:469 read 0x59c720c thru 0x59c7f31 (0xd26 bytes) 2024-09-16T11:08:30.637 INFO:teuthology.orchestra.run.smithi102.stdout:470 write 0x1adf27f thru 0x1ae417f (0x4f01 bytes) 2024-09-16T11:08:30.726 INFO:teuthology.orchestra.run.smithi102.stdout:471 trunc from 0x742fec9 to 0x27917b1 2024-09-16T11:08:31.058 INFO:teuthology.orchestra.run.smithi102.stdout:472 punch from 0x1780942 to 0x178d342, (0xca00 bytes) 2024-09-16T11:08:31.059 INFO:teuthology.orchestra.run.smithi102.stdout:473 read 0x146d6c3 thru 0x1473e0c (0x674a bytes) 2024-09-16T11:08:31.074 INFO:teuthology.orchestra.run.smithi102.stdout:474 trunc from 0x27917b1 to 0xb70974 2024-09-16T11:08:31.526 INFO:teuthology.orchestra.run.smithi102.stdout:475 trunc from 0xb70974 to 0x3b6f29d 2024-09-16T11:08:31.555 INFO:teuthology.orchestra.run.smithi102.stdout:476 trunc from 0x3b6f29d to 0x5c5be6b 2024-09-16T11:08:31.569 INFO:teuthology.orchestra.run.smithi102.stdout:478 read 0x40c3692 thru 0x40d195a (0xe2c9 bytes) 2024-09-16T11:08:31.570 INFO:teuthology.orchestra.run.smithi102.stdout:479 read 0xff3686 thru 0xff96ca (0x6045 bytes) 2024-09-16T11:08:31.572 INFO:teuthology.orchestra.run.smithi102.stdout:480 read 0x297c167 thru 0x298afea (0xee84 bytes) 2024-09-16T11:08:31.573 INFO:teuthology.orchestra.run.smithi102.stdout:482 write 0xc9f83e thru 0xca0279 (0xa3c bytes) 2024-09-16T11:08:31.575 INFO:teuthology.orchestra.run.smithi102.stdout:483 read 0x2e1d5ca thru 0x2e211d0 (0x3c07 bytes) 2024-09-16T11:08:31.576 INFO:teuthology.orchestra.run.smithi102.stdout:compare and write disabled 2024-09-16T11:08:31.577 INFO:teuthology.orchestra.run.smithi102.stdout:486 write 0xe732f4 thru 0xe82c84 (0xf991 bytes) 2024-09-16T11:08:31.582 INFO:teuthology.orchestra.run.smithi102.stdout:487 write 0x7f7fad thru 0x7fd2ad (0x5301 bytes) 2024-09-16T11:08:31.584 INFO:teuthology.orchestra.run.smithi102.stdout:488 read 0x185ae33 thru 0x185bfbf (0x118d bytes) 2024-09-16T11:08:31.587 INFO:teuthology.orchestra.run.smithi102.stdout:489 writesame 0x120180b thru 0x1203f39 data_size 0x1(0x272f bytes) 2024-09-16T11:08:31.592 INFO:teuthology.orchestra.run.smithi102.stdout:490 punch from 0x335e9a0 to 0x3364d65, (0x63c5 bytes) 2024-09-16T11:08:31.597 INFO:teuthology.orchestra.run.smithi102.stdout:491 writesame 0x51ca0f9 thru 0x51d4315 data_size 0x1(0xa21d bytes) 2024-09-16T11:08:31.642 INFO:teuthology.orchestra.run.smithi102.stdout:492 write 0x6a24c46 thru 0x6a34a5a (0xfe15 bytes) 2024-09-16T11:08:31.647 INFO:teuthology.orchestra.run.smithi102.stdout:495 clone 4 order 22 su 2097152 sc 14 2024-09-16T11:08:32.235 INFO:teuthology.orchestra.run.smithi102.stdout:truncating image image_client.0-clone3 from 0x6a34a5b (overlap 0xb70974) to 0x0 2024-09-16T11:08:32.729 INFO:teuthology.orchestra.run.smithi102.stdout:checking clone #2, image image_client.0-clone2 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent3 2024-09-16T11:08:32.822 INFO:teuthology.orchestra.run.smithi102.stdout:496 read 0x369d45f thru 0x369e396 (0xf38 bytes) 2024-09-16T11:08:32.823 INFO:teuthology.orchestra.run.smithi102.stdout:497 punch from 0x3bc529b to 0x3bc5481, (0x1e6 bytes) 2024-09-16T11:08:32.830 INFO:teuthology.orchestra.run.smithi102.stdout:499 write 0x24515df thru 0x2455ae1 (0x4503 bytes) 2024-09-16T11:08:32.869 INFO:teuthology.orchestra.run.smithi102.stdout:500 write 0x78d2945 thru 0x78e15b8 (0xec74 bytes) 2024-09-16T11:08:32.876 INFO:teuthology.orchestra.run.smithi102.stdout:503 write 0x55c8cf3 thru 0x55d6e2c (0xe13a bytes) 2024-09-16T11:08:32.885 INFO:teuthology.orchestra.run.smithi102.stdout:504 write 0x8a51a3 thru 0x8b336f (0xe1cd bytes) 2024-09-16T11:08:32.889 INFO:teuthology.orchestra.run.smithi102.stdout:505 punch from 0x9d824e to 0x9e2bda, (0xa98c bytes) 2024-09-16T11:08:32.889 INFO:teuthology.orchestra.run.smithi102.stdout:507 writesame 0x51fce57 thru 0x52062bb data_size 0x51(0x9465 bytes) 2024-09-16T11:08:32.906 INFO:teuthology.orchestra.run.smithi102.stdout:508 punch from 0x66588c2 to 0x665eec3, (0x6601 bytes) 2024-09-16T11:08:32.906 INFO:teuthology.orchestra.run.smithi102.stdout:509 read 0x1bedc1 thru 0x1ce5a1 (0xf7e1 bytes) 2024-09-16T11:08:32.910 INFO:teuthology.orchestra.run.smithi102.stdout:510 punch from 0x6cd2a02 to 0x6cd72c1, (0x48bf bytes) 2024-09-16T11:08:32.917 INFO:teuthology.orchestra.run.smithi102.stdout:511 writesame 0x5bbfb52 thru 0x5bcd91c data_size 0x1(0xddcb bytes) 2024-09-16T11:08:32.949 INFO:teuthology.orchestra.run.smithi102.stdout:compare and write disabled 2024-09-16T11:08:32.950 INFO:teuthology.orchestra.run.smithi102.stdout:515 read 0x125ffb0 thru 0x126326b (0x32bc bytes) 2024-09-16T11:08:32.989 INFO:teuthology.orchestra.run.smithi102.stdout:517 trunc from 0x78e15b9 to 0x1eb07ac 2024-09-16T11:08:33.125 INFO:teuthology.orchestra.run.smithi102.stdout:518 write 0x166c9a6 thru 0x166d1e3 (0x83e bytes) 2024-09-16T11:08:33.127 INFO:teuthology.orchestra.run.smithi102.stdout:521 write 0x1e673a1 thru 0x1e6c20e (0x4e6e bytes) 2024-09-16T11:08:33.130 INFO:teuthology.orchestra.run.smithi102.stdout:522 read 0x1a4dd30 thru 0x1a51cc4 (0x3f95 bytes) 2024-09-16T11:08:33.133 INFO:teuthology.orchestra.run.smithi102.stdout:524 writesame 0x1815103 thru 0x1817167 data_size 0x1(0x2065 bytes) 2024-09-16T11:08:33.187 INFO:teuthology.orchestra.run.smithi102.stdout:529 write 0x6e461ea thru 0x6e4e1cd (0x7fe4 bytes) 2024-09-16T11:08:33.192 INFO:teuthology.orchestra.run.smithi102.stdout:532 write 0x41fdaf7 thru 0x42038d3 (0x5ddd bytes) 2024-09-16T11:08:33.201 INFO:teuthology.orchestra.run.smithi102.stdout:533 write 0x3f44426 thru 0x3f4819e (0x3d79 bytes) 2024-09-16T11:08:33.206 INFO:teuthology.orchestra.run.smithi102.stdout:534 read 0x2317ecf thru 0x231a873 (0x29a5 bytes) 2024-09-16T11:08:33.207 INFO:teuthology.orchestra.run.smithi102.stdout:compare and write disabled 2024-09-16T11:08:33.208 INFO:teuthology.orchestra.run.smithi102.stdout:539 flatten 2024-09-16T11:08:33.300 INFO:teuthology.orchestra.run.smithi102.stdout:540 writesame 0x35f5d12 thru 0x35fd890 data_size 0x1(0x7b7f bytes) 2024-09-16T11:08:33.316 INFO:teuthology.orchestra.run.smithi102.stdout:543 write 0x6ca9ddd thru 0x6cb8c55 (0xee79 bytes) 2024-09-16T11:08:33.338 INFO:teuthology.orchestra.run.smithi102.stdout:544 write 0x7162598 thru 0x7171c7b (0xf6e4 bytes) 2024-09-16T11:08:33.344 INFO:teuthology.orchestra.run.smithi102.stdout:545 write 0x607c94b thru 0x607cdc9 (0x47f bytes) 2024-09-16T11:08:33.348 INFO:teuthology.orchestra.run.smithi102.stdout:546 write 0x463d444 thru 0x4642a5d (0x561a bytes) 2024-09-16T11:08:33.353 INFO:teuthology.orchestra.run.smithi102.stdout:547 write 0x22fd649 thru 0x23051b2 (0x7b6a bytes) 2024-09-16T11:08:33.356 INFO:teuthology.orchestra.run.smithi102.stdout:548 write 0x7eb6f6 thru 0x7f470d (0x9018 bytes) 2024-09-16T11:08:33.359 INFO:teuthology.orchestra.run.smithi102.stdout:549 read 0x590be08 thru 0x5918f6e (0xd167 bytes) 2024-09-16T11:08:33.379 INFO:teuthology.orchestra.run.smithi102.stdout:551 write 0x7826cf5 thru 0x7832dbd (0xc0c9 bytes) 2024-09-16T11:08:33.385 INFO:teuthology.orchestra.run.smithi102.stdout:compare and write disabled 2024-09-16T11:08:33.386 INFO:teuthology.orchestra.run.smithi102.stdout:553 write 0x365d383 thru 0x366d349 (0xffc7 bytes) 2024-09-16T11:08:33.389 INFO:teuthology.orchestra.run.smithi102.stdout:554 punch from 0xfa84d9 to 0xfb6ce2, (0xe809 bytes) 2024-09-16T11:08:33.390 INFO:teuthology.orchestra.run.smithi102.stdout:556 read 0x4be8751 thru 0x4bf61a7 (0xda57 bytes) 2024-09-16T11:08:33.391 INFO:teuthology.orchestra.run.smithi102.stdout:558 write 0x19acf63 thru 0x19b15b4 (0x4652 bytes) 2024-09-16T11:08:33.394 INFO:teuthology.orchestra.run.smithi102.stdout:559 punch from 0x2169ac0 to 0x21734d2, (0x9a12 bytes) 2024-09-16T11:08:33.394 INFO:teuthology.orchestra.run.smithi102.stdout:561 read 0x3aa23e5 thru 0x3aac8e1 (0xa4fd bytes) 2024-09-16T11:08:33.396 INFO:teuthology.orchestra.run.smithi102.stdout:563 writesame 0x74c3dff thru 0x74cd28c data_size 0x5(0x948e bytes) 2024-09-16T11:08:33.404 INFO:teuthology.orchestra.run.smithi102.stdout:565 writesame 0x6732bcf thru 0x673b8c9 data_size 0x11(0x8cfb bytes) 2024-09-16T11:08:33.410 INFO:teuthology.orchestra.run.smithi102.stdout:compare and write disabled 2024-09-16T11:08:33.410 INFO:teuthology.orchestra.run.smithi102.stdout:569 punch from 0x16713dc to 0x167964a, (0x826e bytes) 2024-09-16T11:08:33.410 INFO:teuthology.orchestra.run.smithi102.stdout:570 punch from 0x42e5905 to 0x42eebf3, (0x92ee bytes) 2024-09-16T11:08:33.411 INFO:teuthology.orchestra.run.smithi102.stdout:571 write 0x3d44da0 thru 0x3d50322 (0xb583 bytes) 2024-09-16T11:08:33.415 INFO:teuthology.orchestra.run.smithi102.stdout:572 read 0x1a34495 thru 0x1a43b7d (0xf6e9 bytes) 2024-09-16T11:08:33.419 INFO:teuthology.orchestra.run.smithi102.stdout:575 write 0x5ab695c thru 0x5abb07b (0x4720 bytes) 2024-09-16T11:08:33.421 INFO:teuthology.orchestra.run.smithi102.stdout:577 clone 5 order 23 su 4194304 sc 14 2024-09-16T11:08:34.467 INFO:teuthology.orchestra.run.smithi102.stdout:checking clone #3, image image_client.0-clone3 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent4 2024-09-16T11:08:34.476 INFO:teuthology.orchestra.run.smithi102.stdout:578 write 0x532e8ce thru 0x533ba2c (0xd15f bytes) 2024-09-16T11:08:34.495 INFO:teuthology.orchestra.run.smithi102.stdout:579 punch from 0x1c88e30 to 0x1c8edff, (0x5fcf bytes) 2024-09-16T11:08:34.495 INFO:teuthology.orchestra.run.smithi102.stdout:580 punch from 0x2534ecc to 0x253b8c8, (0x69fc bytes) 2024-09-16T11:08:34.546 INFO:teuthology.orchestra.run.smithi102.stdout:581 trunc from 0x7832dbe to 0x30a360 2024-09-16T11:08:34.629 INFO:teuthology.orchestra.run.smithi102.stdout:582 write 0x2351d81 thru 0x23609b6 (0xec36 bytes) 2024-09-16T11:08:34.635 INFO:teuthology.orchestra.run.smithi102.stdout:583 punch from 0x1cd34af to 0x1cd8c68, (0x57b9 bytes) 2024-09-16T11:08:34.660 INFO:teuthology.orchestra.run.smithi102.stdout:584 writesame 0x3abe0ad thru 0x3ac39b4 data_size 0x58(0x5908 bytes) 2024-09-16T11:08:34.663 INFO:teuthology.orchestra.run.smithi102.stdout:585 writesame 0x31def94 thru 0x31e29b5 data_size 0x7(0x3a22 bytes) 2024-09-16T11:08:34.699 INFO:teuthology.orchestra.run.smithi102.stdout:586 write 0x6036c49 thru 0x603be94 (0x524c bytes) 2024-09-16T11:08:34.704 INFO:teuthology.orchestra.run.smithi102.stdout:587 punch from 0x1372d35 to 0x1376ded, (0x40b8 bytes) 2024-09-16T11:08:34.717 INFO:teuthology.orchestra.run.smithi102.stdout:588 trunc from 0x603be95 to 0x78ec529 2024-09-16T11:08:34.731 INFO:teuthology.orchestra.run.smithi102.stdout:589 read 0x6761367 thru 0x676324e (0x1ee8 bytes) 2024-09-16T11:08:34.732 INFO:teuthology.orchestra.run.smithi102.stdout:compare and write disabled 2024-09-16T11:08:34.732 INFO:teuthology.orchestra.run.smithi102.stdout:591 punch from 0x2638600 to 0x263b442, (0x2e42 bytes) 2024-09-16T11:08:34.732 INFO:teuthology.orchestra.run.smithi102.stdout:593 writesame 0x5639d5b thru 0x5643b4a data_size 0x4c(0x9df0 bytes) 2024-09-16T11:08:34.737 INFO:teuthology.orchestra.run.smithi102.stdout:compare and write disabled 2024-09-16T11:08:34.743 INFO:teuthology.orchestra.run.smithi102.stdout:597 writesame 0x98b05e thru 0x997b68 data_size 0x1(0xcb0b bytes) 2024-09-16T11:08:34.772 INFO:teuthology.orchestra.run.smithi102.stdout:598 writesame 0x53dd80a thru 0x53eb81f data_size 0x9(0xe016 bytes) 2024-09-16T11:08:34.780 INFO:teuthology.orchestra.run.smithi102.stdout:compare and write disabled 2024-09-16T11:08:34.780 INFO:teuthology.orchestra.run.smithi102.stdout:602 writesame 0x2152290 thru 0x215b055 data_size 0x45(0x8dc6 bytes) 2024-09-16T11:08:34.782 INFO:teuthology.orchestra.run.smithi102.stdout:603 read 0xacd27c thru 0xad58c2 (0x8647 bytes) 2024-09-16T11:08:34.784 INFO:teuthology.orchestra.run.smithi102.stdout:604 punch from 0x3e94cb6 to 0x3e973d2, (0x271c bytes) 2024-09-16T11:08:34.785 INFO:teuthology.orchestra.run.smithi102.stdout:compare and write disabled 2024-09-16T11:08:34.785 INFO:teuthology.orchestra.run.smithi102.stdout:compare and write disabled 2024-09-16T11:08:34.785 INFO:teuthology.orchestra.run.smithi102.stdout:compare and write disabled 2024-09-16T11:08:34.785 INFO:teuthology.orchestra.run.smithi102.stdout:compare and write disabled 2024-09-16T11:08:34.785 INFO:teuthology.orchestra.run.smithi102.stdout:611 punch from 0x2ccb060 to 0x2cda5d8, (0xf578 bytes) 2024-09-16T11:08:34.801 INFO:teuthology.orchestra.run.smithi102.stdout:613 write 0x7d6195d thru 0x7d62035 (0x6d9 bytes) 2024-09-16T11:08:34.805 INFO:teuthology.orchestra.run.smithi102.stdout:614 punch from 0x2488ef3 to 0x2490672, (0x777f bytes) 2024-09-16T11:08:34.819 INFO:teuthology.orchestra.run.smithi102.stdout:616 trunc from 0x7d62036 to 0x5cfca7e 2024-09-16T11:08:34.843 INFO:teuthology.orchestra.run.smithi102.stdout:617 write 0x2ea4e99 thru 0x2eac777 (0x78df bytes) 2024-09-16T11:08:34.876 INFO:teuthology.orchestra.run.smithi102.stdout:618 trunc from 0x5cfca7e to 0x1bcde04 2024-09-16T11:08:34.914 INFO:teuthology.orchestra.run.smithi102.stdout:619 read 0x14df83a thru 0x14e7b82 (0x8349 bytes) 2024-09-16T11:08:34.915 INFO:teuthology.orchestra.run.smithi102.stdout:622 read 0xf6182e thru 0xf61e63 (0x636 bytes) 2024-09-16T11:08:34.915 INFO:teuthology.orchestra.run.smithi102.stdout:623 read 0x119db83 thru 0x11a008e (0x250c bytes) 2024-09-16T11:08:34.949 INFO:teuthology.orchestra.run.smithi102.stdout:624 write 0x45a1f53 thru 0x45a214b (0x1f9 bytes) 2024-09-16T11:08:34.972 INFO:teuthology.orchestra.run.smithi102.stdout:626 trunc from 0x45a214c to 0x18ce3a3 2024-09-16T11:08:35.015 INFO:teuthology.orchestra.run.smithi102.stdout:629 trunc from 0x18ce3a3 to 0x4df4873 2024-09-16T11:08:35.029 INFO:teuthology.orchestra.run.smithi102.stdout:compare and write disabled 2024-09-16T11:08:35.030 INFO:teuthology.orchestra.run.smithi102.stdout:631 read 0x204aa99 thru 0x20522d5 (0x783d bytes) 2024-09-16T11:08:35.030 INFO:teuthology.orchestra.run.smithi102.stdout:632 read 0x1fff65c thru 0x20085c6 (0x8f6b bytes) 2024-09-16T11:08:35.032 INFO:teuthology.orchestra.run.smithi102.stdout:633 trunc from 0x4df4873 to 0x5067104 2024-09-16T11:08:35.048 INFO:teuthology.orchestra.run.smithi102.stdout:635 trunc from 0x5067104 to 0x5599af3 2024-09-16T11:08:35.062 INFO:teuthology.orchestra.run.smithi102.stdout:638 read 0x1667eb9 thru 0x166d7bb (0x5903 bytes) 2024-09-16T11:08:35.063 INFO:teuthology.orchestra.run.smithi102.stdout:compare and write disabled 2024-09-16T11:08:35.064 INFO:teuthology.orchestra.run.smithi102.stdout:640 read 0x2e14237 thru 0x2e1a727 (0x64f1 bytes) 2024-09-16T11:08:35.064 INFO:teuthology.orchestra.run.smithi102.stdout:641 read 0x399f294 thru 0x39a910c (0x9e79 bytes) 2024-09-16T11:08:35.066 INFO:teuthology.orchestra.run.smithi102.stdout:643 write 0x2839830 thru 0x283f91c (0x60ed bytes) 2024-09-16T11:08:35.071 INFO:teuthology.orchestra.run.smithi102.stdout:644 write 0x36993f2 thru 0x36a4ca3 (0xb8b2 bytes) 2024-09-16T11:08:35.109 INFO:teuthology.orchestra.run.smithi102.stdout:646 trunc from 0x5599af3 to 0x9fc0e4 2024-09-16T11:08:35.145 INFO:teuthology.orchestra.run.smithi102.stdout:compare and write disabled 2024-09-16T11:08:35.145 INFO:teuthology.orchestra.run.smithi102.stdout:650 read 0x9ea2b6 thru 0x9eecc6 (0x4a11 bytes) 2024-09-16T11:08:35.145 INFO:teuthology.orchestra.run.smithi102.stdout:651 punch from 0x2172ae to 0x21d807, (0x6559 bytes) 2024-09-16T11:08:35.145 INFO:teuthology.orchestra.run.smithi102.stdout:compare and write disabled 2024-09-16T11:08:35.173 INFO:teuthology.orchestra.run.smithi102.stdout:653 writesame 0x237911b thru 0x237dc27 data_size 0x1(0x4b0d bytes) 2024-09-16T11:08:35.173 INFO:teuthology.orchestra.run.smithi102.stdout:compare and write disabled 2024-09-16T11:08:35.192 INFO:teuthology.orchestra.run.smithi102.stdout:655 trunc from 0x237dc28 to 0x4f4e79a 2024-09-16T11:08:35.241 INFO:teuthology.orchestra.run.smithi102.stdout:656 write 0x7b5e2df thru 0x7b6b154 (0xce76 bytes) 2024-09-16T11:08:35.248 INFO:teuthology.orchestra.run.smithi102.stdout:compare and write disabled 2024-09-16T11:08:35.248 INFO:teuthology.orchestra.run.smithi102.stdout:compare and write disabled 2024-09-16T11:08:35.260 INFO:teuthology.orchestra.run.smithi102.stdout:660 trunc from 0x7b6b155 to 0x5d2bad8 2024-09-16T11:08:35.282 INFO:teuthology.orchestra.run.smithi102.stdout:661 write 0xa740df thru 0xa7ccbe (0x8be0 bytes) 2024-09-16T11:08:35.285 INFO:teuthology.orchestra.run.smithi102.stdout:662 write 0x139f39d thru 0x13aa5a2 (0xb206 bytes) 2024-09-16T11:08:35.291 INFO:teuthology.orchestra.run.smithi102.stdout:663 write 0x4da6492 thru 0x4dadc10 (0x777f bytes) 2024-09-16T11:08:35.296 INFO:teuthology.orchestra.run.smithi102.stdout:664 read 0x3b0a1f2 thru 0x3b0e129 (0x3f38 bytes) 2024-09-16T11:08:35.303 INFO:teuthology.orchestra.run.smithi102.stdout:665 trunc from 0x5d2bad8 to 0x6ad347e 2024-09-16T11:08:35.318 INFO:teuthology.orchestra.run.smithi102.stdout:666 read 0x5b21d20 thru 0x5b2a5e5 (0x88c6 bytes) 2024-09-16T11:08:35.319 INFO:teuthology.orchestra.run.smithi102.stdout:667 write 0x56d8191 thru 0x56e20ab (0x9f1b bytes) 2024-09-16T11:08:35.364 INFO:teuthology.orchestra.run.smithi102.stdout:668 trunc from 0x6ad347e to 0x11ae2f2 2024-09-16T11:08:35.389 INFO:teuthology.orchestra.run.smithi102.stdout:669 read 0x122728 thru 0x12b2c0 (0x8b99 bytes) 2024-09-16T11:08:35.413 INFO:teuthology.orchestra.run.smithi102.stdout:670 writesame 0x154bea8 thru 0x1557440 data_size 0x1(0xb599 bytes) 2024-09-16T11:08:35.414 INFO:teuthology.orchestra.run.smithi102.stdout:compare and write disabled 2024-09-16T11:08:35.414 INFO:teuthology.orchestra.run.smithi102.stdout:compare and write disabled 2024-09-16T11:08:35.414 INFO:teuthology.orchestra.run.smithi102.stdout:677 punch from 0xc94bd3 to 0xc9527d, (0x6aa bytes) 2024-09-16T11:08:35.455 INFO:teuthology.orchestra.run.smithi102.stdout:679 writesame 0x4b25b21 thru 0x4b292a9 data_size 0x7(0x3789 bytes) 2024-09-16T11:08:35.487 INFO:teuthology.orchestra.run.smithi102.stdout:680 write 0x6f216c9 thru 0x6f2da8a (0xc3c2 bytes) 2024-09-16T11:08:35.493 INFO:teuthology.orchestra.run.smithi102.stdout:681 trunc from 0x6f2da8b to 0x6fbb410 2024-09-16T11:08:35.507 INFO:teuthology.orchestra.run.smithi102.stdout:682 punch from 0x648ddb7 to 0x64908d5, (0x2b1e bytes) 2024-09-16T11:08:35.513 INFO:teuthology.orchestra.run.smithi102.stdout:683 trunc from 0x6fbb410 to 0x635f574 2024-09-16T11:08:35.530 INFO:teuthology.orchestra.run.smithi102.stdout:685 read 0x40243e9 thru 0x4029e09 (0x5a21 bytes) 2024-09-16T11:08:35.532 INFO:teuthology.orchestra.run.smithi102.stdout:686 write 0x215f42f thru 0x2167604 (0x81d6 bytes) 2024-09-16T11:08:35.539 INFO:teuthology.orchestra.run.smithi102.stdout:687 write 0x5f17d39 thru 0x5f27adf (0xfda7 bytes) 2024-09-16T11:08:35.544 INFO:teuthology.orchestra.run.smithi102.stdout:688 read 0x9a0f7b thru 0x9b01c2 (0xf248 bytes) 2024-09-16T11:08:35.547 INFO:teuthology.orchestra.run.smithi102.stdout:689 punch from 0x3ce69f1 to 0x3cf5152, (0xe761 bytes) 2024-09-16T11:08:35.548 INFO:teuthology.orchestra.run.smithi102.stdout:690 punch from 0x289821d to 0x28a73b0, (0xf193 bytes) 2024-09-16T11:08:35.548 INFO:teuthology.orchestra.run.smithi102.stdout:691 read 0x4476fad thru 0x4477c42 (0xc96 bytes) 2024-09-16T11:08:35.548 INFO:teuthology.orchestra.run.smithi102.stdout:692 clone 6 order 21 su 262144 sc 15 2024-09-16T11:08:36.132 INFO:teuthology.orchestra.run.smithi102.stdout:leaving image image_client.0-clone5 intact 2024-09-16T11:08:36.540 INFO:teuthology.orchestra.run.smithi102.stdout:checking clone #4, image image_client.0-clone4 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent5 2024-09-16T11:08:38.669 INFO:teuthology.orchestra.run.smithi102.stdout:compare and write disabled 2024-09-16T11:08:38.671 INFO:teuthology.orchestra.run.smithi102.stdout:694 writesame 0x2f79ea9 thru 0x2f7ea67 data_size 0x1(0x4bbf bytes) 2024-09-16T11:08:38.698 INFO:teuthology.orchestra.run.smithi102.stdout:695 write 0x32a0ac1 thru 0x32ab4fa (0xaa3a bytes) 2024-09-16T11:08:38.708 INFO:teuthology.orchestra.run.smithi102.stdout:697 write 0x31f7889 thru 0x32011d5 (0x994d bytes) 2024-09-16T11:08:38.722 INFO:teuthology.orchestra.run.smithi102.stdout:698 read 0x4476ba7 thru 0x447e0c6 (0x7520 bytes) 2024-09-16T11:08:38.723 INFO:teuthology.orchestra.run.smithi102.stdout:compare and write disabled 2024-09-16T11:08:38.723 INFO:teuthology.orchestra.run.smithi102.stdout:700 writesame 0x6c2044 thru 0x6d099f data_size 0x74(0xe95c bytes) 2024-09-16T11:08:38.731 INFO:teuthology.orchestra.run.smithi102.stdout:701 read 0x547ed60 thru 0x5487304 (0x85a5 bytes) 2024-09-16T11:08:38.740 INFO:teuthology.orchestra.run.smithi102.stdout:702 trunc from 0x635f574 to 0x5463f93 2024-09-16T11:08:38.842 INFO:teuthology.orchestra.run.smithi102.stdout:704 writesame 0x715fb67 thru 0x7163934 data_size 0x1b(0x3dce bytes) 2024-09-16T11:08:38.852 INFO:teuthology.orchestra.run.smithi102.stdout:706 trunc from 0x7163935 to 0x5bfa063 2024-09-16T11:08:38.869 INFO:teuthology.orchestra.run.smithi102.stdout:compare and write disabled 2024-09-16T11:08:38.869 INFO:teuthology.orchestra.run.smithi102.stdout:709 write 0x231fadb thru 0x232eeee (0xf414 bytes) 2024-09-16T11:08:38.902 INFO:teuthology.orchestra.run.smithi102.stdout:711 write 0x7c06be1 thru 0x7c0fcc4 (0x90e4 bytes) 2024-09-16T11:08:38.907 INFO:teuthology.orchestra.run.smithi102.stdout:712 read 0x3f66bad thru 0x3f6bc2e (0x5082 bytes) 2024-09-16T11:08:38.909 INFO:teuthology.orchestra.run.smithi102.stdout:713 read 0x75ea7a thru 0x76d561 (0xeae8 bytes) 2024-09-16T11:08:38.911 INFO:teuthology.orchestra.run.smithi102.stdout:714 write 0x41a7963 thru 0x41b04ff (0x8b9d bytes) 2024-09-16T11:08:38.918 INFO:teuthology.orchestra.run.smithi102.stdout:715 trunc from 0x7c0fcc5 to 0x72b14b6 2024-09-16T11:08:38.941 INFO:teuthology.orchestra.run.smithi102.stdout:718 read 0x1d954c3 thru 0x1d9b055 (0x5b93 bytes) 2024-09-16T11:08:38.943 INFO:teuthology.orchestra.run.smithi102.stdout:720 read 0x15ed8f5 thru 0x15fc383 (0xea8f bytes) 2024-09-16T11:08:38.944 INFO:teuthology.orchestra.run.smithi102.stdout:compare and write disabled 2024-09-16T11:08:38.945 INFO:teuthology.orchestra.run.smithi102.stdout:722 read 0x48c5cc5 thru 0x48cbc4d (0x5f89 bytes) 2024-09-16T11:08:38.974 INFO:teuthology.orchestra.run.smithi102.stdout:726 trunc from 0x72b14b6 to 0x320dd11 2024-09-16T11:08:39.064 INFO:teuthology.orchestra.run.smithi102.stdout:compare and write disabled 2024-09-16T11:08:39.064 INFO:teuthology.orchestra.run.smithi102.stdout:compare and write disabled 2024-09-16T11:08:39.064 INFO:teuthology.orchestra.run.smithi102.stdout:compare and write disabled 2024-09-16T11:08:39.064 INFO:teuthology.orchestra.run.smithi102.stdout:731 read 0x192752d thru 0x1931e53 (0xa927 bytes) 2024-09-16T11:08:39.065 INFO:teuthology.orchestra.run.smithi102.stdout:732 punch from 0x220c023 to 0x2217edc, (0xbeb9 bytes) 2024-09-16T11:08:39.065 INFO:teuthology.orchestra.run.smithi102.stdout:733 write 0x1131a7f thru 0x1134049 (0x25cb bytes) 2024-09-16T11:08:39.075 INFO:teuthology.orchestra.run.smithi102.stdout:734 write 0xc0cee4 thru 0xc13868 (0x6985 bytes) 2024-09-16T11:08:39.112 INFO:teuthology.orchestra.run.smithi102.stdout:735 writesame 0x4739cc6 thru 0x4742242 data_size 0x3(0x857d bytes) 2024-09-16T11:08:39.113 INFO:teuthology.orchestra.run.smithi102.stdout:compare and write disabled 2024-09-16T11:08:39.113 INFO:teuthology.orchestra.run.smithi102.stdout:737 write 0x3b0d418 thru 0x3b133fb (0x5fe4 bytes) 2024-09-16T11:08:39.116 INFO:teuthology.orchestra.run.smithi102.stdout:738 read 0x18b831 thru 0x1968c8 (0xb098 bytes) 2024-09-16T11:08:39.120 INFO:teuthology.orchestra.run.smithi102.stdout:741 write 0x3ca0e5c thru 0x3cb0874 (0xfa19 bytes) 2024-09-16T11:08:39.125 INFO:teuthology.orchestra.run.smithi102.stdout:742 read 0x3d17056 thru 0x3d1b600 (0x45ab bytes) 2024-09-16T11:08:39.125 INFO:teuthology.orchestra.run.smithi102.stdout:743 read 0x27d839d thru 0x27e68dc (0xe540 bytes) 2024-09-16T11:08:39.164 INFO:teuthology.orchestra.run.smithi102.stdout:744 writesame 0x75a791f thru 0x75ab590 data_size 0x3(0x3c72 bytes) 2024-09-16T11:08:39.165 INFO:teuthology.orchestra.run.smithi102.stdout:745 trunc from 0x75ab591 to 0x72a0e49 2024-09-16T11:08:39.181 INFO:teuthology.orchestra.run.smithi102.stdout:746 read 0x599585a thru 0x59a134c (0xbaf3 bytes) 2024-09-16T11:08:39.182 INFO:teuthology.orchestra.run.smithi102.stdout:747 read 0x4f6ab53 thru 0x4f77a53 (0xcf01 bytes) 2024-09-16T11:08:39.184 INFO:teuthology.orchestra.run.smithi102.stdout:compare and write disabled 2024-09-16T11:08:39.184 INFO:teuthology.orchestra.run.smithi102.stdout:750 read 0x51005e3 thru 0x510d832 (0xd250 bytes) 2024-09-16T11:08:39.185 INFO:teuthology.orchestra.run.smithi102.stdout:751 read 0x10a02a1 thru 0x10aab09 (0xa869 bytes) 2024-09-16T11:08:39.187 INFO:teuthology.orchestra.run.smithi102.stdout:752 read 0x153658 thru 0x15f7ea (0xc193 bytes) 2024-09-16T11:08:39.190 INFO:teuthology.orchestra.run.smithi102.stdout:753 read 0x60b7088 thru 0x60c5ff9 (0xef72 bytes) 2024-09-16T11:08:39.192 INFO:teuthology.orchestra.run.smithi102.stdout:754 write 0x2ccfce4 thru 0x2cd6001 (0x631e bytes) 2024-09-16T11:08:39.195 INFO:teuthology.orchestra.run.smithi102.stdout:757 read 0x5653f61 thru 0x5661410 (0xd4b0 bytes) 2024-09-16T11:08:39.196 INFO:teuthology.orchestra.run.smithi102.stdout:760 writesame 0x1f64f06 thru 0x1f6d171 data_size 0x22(0x826c bytes) 2024-09-16T11:08:39.199 INFO:teuthology.orchestra.run.smithi102.stdout:761 read 0x367b798 thru 0x36837c5 (0x802e bytes) 2024-09-16T11:08:39.201 INFO:teuthology.orchestra.run.smithi102.stdout:762 read 0x6059731 thru 0x605e28f (0x4b5f bytes) 2024-09-16T11:08:39.239 INFO:teuthology.orchestra.run.smithi102.stdout:763 trunc from 0x72a0e49 to 0x1d43979 2024-09-16T11:08:39.337 INFO:teuthology.orchestra.run.smithi102.stdout:764 writesame 0x74ba1d0 thru 0x74c30b0 data_size 0x1(0x8ee1 bytes) 2024-09-16T11:08:39.338 INFO:teuthology.orchestra.run.smithi102.stdout:765 write 0x649d272 thru 0x64a6952 (0x96e1 bytes) 2024-09-16T11:08:39.343 INFO:teuthology.orchestra.run.smithi102.stdout:766 write 0x5266a58 thru 0x526ec5f (0x8208 bytes) 2024-09-16T11:08:39.348 INFO:teuthology.orchestra.run.smithi102.stdout:768 punch from 0x489c983 to 0x48a5cee, (0x936b bytes) 2024-09-16T11:08:39.375 INFO:teuthology.orchestra.run.smithi102.stdout:769 trunc from 0x74c30b1 to 0x35e0eca 2024-09-16T11:08:39.398 INFO:teuthology.orchestra.run.smithi102.stdout:770 writesame 0x577755 thru 0x582808 data_size 0x56(0xb0b4 bytes) 2024-09-16T11:08:39.454 INFO:teuthology.orchestra.run.smithi102.stdout:771 write 0x6f897ae thru 0x6f92490 (0x8ce3 bytes) 2024-09-16T11:08:39.459 INFO:teuthology.orchestra.run.smithi102.stdout:774 write 0x624a5b4 thru 0x624b98d (0x13da bytes) 2024-09-16T11:08:39.464 INFO:teuthology.orchestra.run.smithi102.stdout:compare and write disabled 2024-09-16T11:08:39.464 INFO:teuthology.orchestra.run.smithi102.stdout:777 punch from 0x6b7610c to 0x6b81d02, (0xbbf6 bytes) 2024-09-16T11:08:39.464 INFO:teuthology.orchestra.run.smithi102.stdout:778 writesame 0x21e45f7 thru 0x21e61e0 data_size 0x9(0x1bea bytes) 2024-09-16T11:08:39.494 INFO:teuthology.orchestra.run.smithi102.stdout:779 writesame 0x7db0acb thru 0x7dbd4cd data_size 0x1(0xca03 bytes) 2024-09-16T11:08:39.495 INFO:teuthology.orchestra.run.smithi102.stdout:780 punch from 0x574eceb to 0x57561cb, (0x74e0 bytes) 2024-09-16T11:08:39.495 INFO:teuthology.orchestra.run.smithi102.stdout:compare and write disabled 2024-09-16T11:08:39.496 INFO:teuthology.orchestra.run.smithi102.stdout:782 punch from 0x544dc2 to 0x54f8c1, (0xaaff bytes) 2024-09-16T11:08:39.496 INFO:teuthology.orchestra.run.smithi102.stdout:compare and write disabled 2024-09-16T11:08:39.496 INFO:teuthology.orchestra.run.smithi102.stdout:784 punch from 0xafa2c4 to 0xb07508, (0xd244 bytes) 2024-09-16T11:08:39.496 INFO:teuthology.orchestra.run.smithi102.stdout:785 punch from 0x18f8847 to 0x1904bc9, (0xc382 bytes) 2024-09-16T11:08:39.497 INFO:teuthology.orchestra.run.smithi102.stdout:786 write 0xc401ec thru 0xc49840 (0x9655 bytes) 2024-09-16T11:08:39.511 INFO:teuthology.orchestra.run.smithi102.stdout:787 read 0x1c86103 thru 0x1c93bf2 (0xdaf0 bytes) 2024-09-16T11:08:39.512 INFO:teuthology.orchestra.run.smithi102.stdout:788 trunc from 0x7dbd4ce to 0x7e27943 2024-09-16T11:08:39.526 INFO:teuthology.orchestra.run.smithi102.stdout:790 read 0x2a1300d thru 0x2a1c372 (0x9366 bytes) 2024-09-16T11:08:39.527 INFO:teuthology.orchestra.run.smithi102.stdout:compare and write disabled 2024-09-16T11:08:39.528 INFO:teuthology.orchestra.run.smithi102.stdout:792 write 0x294d784 thru 0x295aeeb (0xd768 bytes) 2024-09-16T11:08:39.531 INFO:teuthology.orchestra.run.smithi102.stdout:793 write 0x537c16d thru 0x53817c0 (0x5654 bytes) 2024-09-16T11:08:39.539 INFO:teuthology.orchestra.run.smithi102.stdout:794 write 0x600e175 thru 0x601ad85 (0xcc11 bytes) 2024-09-16T11:08:39.549 INFO:teuthology.orchestra.run.smithi102.stdout:795 writesame 0x3aeba05 thru 0x3af70a3 data_size 0x1(0xb69f bytes) 2024-09-16T11:08:39.576 INFO:teuthology.orchestra.run.smithi102.stdout:796 write 0x1bf4ffd thru 0x1bff733 (0xa737 bytes) 2024-09-16T11:08:39.580 INFO:teuthology.orchestra.run.smithi102.stdout:797 read 0x3780afb thru 0x3780e5a (0x360 bytes) 2024-09-16T11:08:39.581 INFO:teuthology.orchestra.run.smithi102.stdout:798 write 0x30ba9ad thru 0x30c51ac (0xa800 bytes) 2024-09-16T11:08:39.587 INFO:teuthology.orchestra.run.smithi102.stdout:799 read 0x44d69af thru 0x44e0b31 (0xa183 bytes) 2024-09-16T11:08:39.627 INFO:teuthology.orchestra.run.smithi102.stdout:800 trunc from 0x7e27943 to 0x25a3121 2024-09-16T11:08:39.687 INFO:teuthology.orchestra.run.smithi102.stdout:801 write 0x439b7f0 thru 0x43a3f56 (0x8767 bytes) 2024-09-16T11:08:39.692 INFO:teuthology.orchestra.run.smithi102.stdout:802 read 0x1563735 thru 0x15681b1 (0x4a7d bytes) 2024-09-16T11:08:39.693 INFO:teuthology.orchestra.run.smithi102.stdout:compare and write disabled 2024-09-16T11:08:39.708 INFO:teuthology.orchestra.run.smithi102.stdout:805 trunc from 0x43a3f57 to 0x66b13a2 2024-09-16T11:08:39.724 INFO:teuthology.orchestra.run.smithi102.stdout:compare and write disabled 2024-09-16T11:08:39.724 INFO:teuthology.orchestra.run.smithi102.stdout:807 read 0x5e4383d thru 0x5e49579 (0x5d3d bytes) 2024-09-16T11:08:39.741 INFO:teuthology.orchestra.run.smithi102.stdout:808 writesame 0x6cd2d26 thru 0x6cde86e data_size 0x2b(0xbb49 bytes) 2024-09-16T11:08:39.748 INFO:teuthology.orchestra.run.smithi102.stdout:811 writesame 0x3df3cdb thru 0x3e02277 data_size 0x1(0xe59d bytes) 2024-09-16T11:08:39.795 INFO:teuthology.orchestra.run.smithi102.stdout:813 writesame 0x7529477 thru 0x7538dc2 data_size 0xa(0xf94c bytes) 2024-09-16T11:08:39.795 INFO:teuthology.orchestra.run.smithi102.stdout:815 read 0x2831043 thru 0x2837c33 (0x6bf1 bytes) 2024-09-16T11:08:39.797 INFO:teuthology.orchestra.run.smithi102.stdout:818 read 0x39fe335 thru 0x39ff0c0 (0xd8c bytes) 2024-09-16T11:08:39.797 INFO:teuthology.orchestra.run.smithi102.stdout:819 read 0x2afb56e thru 0x2b09dc4 (0xe857 bytes) 2024-09-16T11:08:39.798 INFO:teuthology.orchestra.run.smithi102.stdout:820 punch from 0x649a54d to 0x649e3ca, (0x3e7d bytes) 2024-09-16T11:08:39.799 INFO:teuthology.orchestra.run.smithi102.stdout:821 read 0x5bab881 thru 0x5bb1002 (0x5782 bytes) 2024-09-16T11:08:39.800 INFO:teuthology.orchestra.run.smithi102.stdout:compare and write disabled 2024-09-16T11:08:39.800 INFO:teuthology.orchestra.run.smithi102.stdout:823 writesame 0x1628541 thru 0x1632b58 data_size 0x14(0xa618 bytes) 2024-09-16T11:08:39.803 INFO:teuthology.orchestra.run.smithi102.stdout:825 read 0x159ab2 thru 0x16761f (0xdb6e bytes) 2024-09-16T11:08:39.806 INFO:teuthology.orchestra.run.smithi102.stdout:compare and write disabled 2024-09-16T11:08:39.819 INFO:teuthology.orchestra.run.smithi102.stdout:827 trunc from 0x7538dc3 to 0x5829741 2024-09-16T11:08:39.838 INFO:teuthology.orchestra.run.smithi102.stdout:compare and write disabled 2024-09-16T11:08:39.843 INFO:teuthology.orchestra.run.smithi102.stdout:829 writesame 0xc387f5 thru 0xc43cf1 data_size 0x1(0xb4fd bytes) 2024-09-16T11:08:39.883 INFO:teuthology.orchestra.run.smithi102.stdout:830 write 0x64d120c thru 0x64dee2a (0xdc1f bytes) 2024-09-16T11:08:39.888 INFO:teuthology.orchestra.run.smithi102.stdout:compare and write disabled 2024-09-16T11:08:39.909 INFO:teuthology.orchestra.run.smithi102.stdout:832 write 0x7164826 thru 0x71745e7 (0xfdc2 bytes) 2024-09-16T11:08:39.914 INFO:teuthology.orchestra.run.smithi102.stdout:833 punch from 0x2c323c5 to 0x2c323f1, (0x2c bytes) 2024-09-16T11:08:39.915 INFO:teuthology.orchestra.run.smithi102.stdout:834 punch from 0x3c368c4 to 0x3c427ef, (0xbf2b bytes) 2024-09-16T11:08:39.916 INFO:teuthology.orchestra.run.smithi102.stdout:835 write 0x10295e4 thru 0x103758a (0xdfa7 bytes) 2024-09-16T11:08:39.919 INFO:teuthology.orchestra.run.smithi102.stdout:836 read 0x2e4f98f thru 0x2e5e9ac (0xf01e bytes) 2024-09-16T11:08:39.921 INFO:teuthology.orchestra.run.smithi102.stdout:837 write 0x229be14 thru 0x22a4638 (0x8825 bytes) 2024-09-16T11:08:39.927 INFO:teuthology.orchestra.run.smithi102.stdout:838 write 0xf5b523 thru 0xf689c4 (0xd4a2 bytes) 2024-09-16T11:08:39.940 INFO:teuthology.orchestra.run.smithi102.stdout:839 read 0x5bbc4b thru 0x5bd0a2 (0x1458 bytes) 2024-09-16T11:08:39.941 INFO:teuthology.orchestra.run.smithi102.stdout:840 writesame 0x197f35 thru 0x19b4dc data_size 0x44(0x35a8 bytes) 2024-09-16T11:08:39.944 INFO:teuthology.orchestra.run.smithi102.stdout:842 read 0x31d0191 thru 0x31ddf72 (0xdde2 bytes) 2024-09-16T11:08:39.945 INFO:teuthology.orchestra.run.smithi102.stdout:843 write 0x4fddc85 thru 0x4fe3b09 (0x5e85 bytes) 2024-09-16T11:08:39.950 INFO:teuthology.orchestra.run.smithi102.stdout:844 punch from 0x3383c9e to 0x3393bc5, (0xff27 bytes) 2024-09-16T11:08:39.950 INFO:teuthology.orchestra.run.smithi102.stdout:849 read 0x5bbcac5 thru 0x5bbd9b5 (0xef1 bytes) 2024-09-16T11:08:39.971 INFO:teuthology.orchestra.run.smithi102.stdout:850 write 0x7b504bb thru 0x7b5f076 (0xebbc bytes) 2024-09-16T11:08:39.977 INFO:teuthology.orchestra.run.smithi102.stdout:851 read 0x2d5055f thru 0x2d5646a (0x5f0c bytes) 2024-09-16T11:08:39.977 INFO:teuthology.orchestra.run.smithi102.stdout:852 writesame 0x6c4c609 thru 0x6c4e62c data_size 0x2c(0x2024 bytes) 2024-09-16T11:08:39.979 INFO:teuthology.orchestra.run.smithi102.stdout:853 read 0x516b599 thru 0x516e1cd (0x2c35 bytes) 2024-09-16T11:08:39.980 INFO:teuthology.orchestra.run.smithi102.stdout:854 write 0x172f7ab thru 0x1737173 (0x79c9 bytes) 2024-09-16T11:08:39.989 INFO:teuthology.orchestra.run.smithi102.stdout:856 read 0x4fc1c89 thru 0x4fcfde8 (0xe160 bytes) 2024-09-16T11:08:39.992 INFO:teuthology.orchestra.run.smithi102.stdout:857 trunc from 0x7b5f077 to 0x7b7c01d 2024-09-16T11:08:40.006 INFO:teuthology.orchestra.run.smithi102.stdout:858 read 0x2aca61f thru 0x2ada1de (0xfbc0 bytes) 2024-09-16T11:08:40.007 INFO:teuthology.orchestra.run.smithi102.stdout:compare and write disabled 2024-09-16T11:08:40.007 INFO:teuthology.orchestra.run.smithi102.stdout:860 read 0x1b2635d thru 0x1b30400 (0xa0a4 bytes) 2024-09-16T11:08:40.009 INFO:teuthology.orchestra.run.smithi102.stdout:compare and write disabled 2024-09-16T11:08:40.009 INFO:teuthology.orchestra.run.smithi102.stdout:compare and write disabled 2024-09-16T11:08:40.009 INFO:teuthology.orchestra.run.smithi102.stdout:867 read 0x527775d thru 0x5285cb2 (0xe556 bytes) 2024-09-16T11:08:40.012 INFO:teuthology.orchestra.run.smithi102.stdout:868 write 0x198a32b thru 0x1990ad3 (0x67a9 bytes) 2024-09-16T11:08:40.014 INFO:teuthology.orchestra.run.smithi102.stdout:870 punch from 0x5a18718 to 0x5a2128f, (0x8b77 bytes) 2024-09-16T11:08:40.016 INFO:teuthology.orchestra.run.smithi102.stdout:871 flatten 2024-09-16T11:08:40.062 INFO:teuthology.orchestra.run.smithi102.stdout:872 writesame 0x1b35010 thru 0x1b41cb9 data_size 0x43(0xccaa bytes) 2024-09-16T11:08:40.073 INFO:teuthology.orchestra.run.smithi102.stdout:873 read 0x3b45b0f thru 0x3b4c2b7 (0x67a9 bytes) 2024-09-16T11:08:40.074 INFO:teuthology.orchestra.run.smithi102.stdout:compare and write disabled 2024-09-16T11:08:40.074 INFO:teuthology.orchestra.run.smithi102.stdout:878 read 0x5dca529 thru 0x5dd4328 (0x9e00 bytes) 2024-09-16T11:08:40.075 INFO:teuthology.orchestra.run.smithi102.stdout:879 punch from 0x113c024 to 0x11432e1, (0x72bd bytes) 2024-09-16T11:08:40.075 INFO:teuthology.orchestra.run.smithi102.stdout:880 read 0x5760556 thru 0x576f6fe (0xf1a9 bytes) 2024-09-16T11:08:40.077 INFO:teuthology.orchestra.run.smithi102.stdout:881 read 0x4782cda thru 0x4791f36 (0xf25d bytes) 2024-09-16T11:08:40.094 INFO:teuthology.orchestra.run.smithi102.stdout:884 write 0x7ee6d61 thru 0x7ee9c80 (0x2f20 bytes) 2024-09-16T11:08:40.098 INFO:teuthology.orchestra.run.smithi102.stdout:886 punch from 0x358a31a to 0x359291c, (0x8602 bytes) 2024-09-16T11:08:40.112 INFO:teuthology.orchestra.run.smithi102.stdout:887 trunc from 0x7ee9c81 to 0x621e3d5 2024-09-16T11:08:40.138 INFO:teuthology.orchestra.run.smithi102.stdout:888 write 0x154beef thru 0x1551e91 (0x5fa3 bytes) 2024-09-16T11:08:40.140 INFO:teuthology.orchestra.run.smithi102.stdout:compare and write disabled 2024-09-16T11:08:40.165 INFO:teuthology.orchestra.run.smithi102.stdout:891 write 0x78242d3 thru 0x782b946 (0x7674 bytes) 2024-09-16T11:08:40.170 INFO:teuthology.orchestra.run.smithi102.stdout:compare and write disabled 2024-09-16T11:08:40.170 INFO:teuthology.orchestra.run.smithi102.stdout:893 write 0x6e16390 thru 0x6e17924 (0x1595 bytes) 2024-09-16T11:08:40.174 INFO:teuthology.orchestra.run.smithi102.stdout:compare and write disabled 2024-09-16T11:08:40.175 INFO:teuthology.orchestra.run.smithi102.stdout:895 write 0x1a361dc thru 0x1a45a7c (0xf8a1 bytes) 2024-09-16T11:08:40.180 INFO:teuthology.orchestra.run.smithi102.stdout:compare and write disabled 2024-09-16T11:08:40.180 INFO:teuthology.orchestra.run.smithi102.stdout:897 write 0x40fac9b thru 0x40fc46a (0x17d0 bytes) 2024-09-16T11:08:40.183 INFO:teuthology.orchestra.run.smithi102.stdout:898 write 0x5e436f3 thru 0x5e45e87 (0x2795 bytes) 2024-09-16T11:08:40.229 INFO:teuthology.orchestra.run.smithi102.stdout:899 trunc from 0x782b947 to 0x1861442 2024-09-16T11:08:40.266 INFO:teuthology.orchestra.run.smithi102.stdout:900 read 0x15004d1 thru 0x150f5b0 (0xf0e0 bytes) 2024-09-16T11:08:40.267 INFO:teuthology.orchestra.run.smithi102.stdout:902 punch from 0xa8b1c5 to 0xa8e249, (0x3084 bytes) 2024-09-16T11:08:40.306 INFO:teuthology.orchestra.run.smithi102.stdout:903 writesame 0x4fb7096 thru 0x4fb7cf9 data_size 0x4(0xc64 bytes) 2024-09-16T11:08:40.307 INFO:teuthology.orchestra.run.smithi102.stdout:905 write 0x2a93894 thru 0x2a9cf32 (0x969f bytes) 2024-09-16T11:08:40.313 INFO:teuthology.orchestra.run.smithi102.stdout:compare and write disabled 2024-09-16T11:08:40.313 INFO:teuthology.orchestra.run.smithi102.stdout:908 write 0x158ff0e thru 0x15908db (0x9ce bytes) 2024-09-16T11:08:40.359 INFO:teuthology.orchestra.run.smithi102.stdout:909 read 0x667c46 thru 0x66cb30 (0x4eeb bytes) 2024-09-16T11:08:40.360 INFO:teuthology.orchestra.run.smithi102.stdout:911 punch from 0x114f326 to 0x115d1ce, (0xdea8 bytes) 2024-09-16T11:08:40.361 INFO:teuthology.orchestra.run.smithi102.stdout:914 read 0x2159d30 thru 0x2167c15 (0xdee6 bytes) 2024-09-16T11:08:40.362 INFO:teuthology.orchestra.run.smithi102.stdout:919 clone 7 order 19 su 8192 sc 8 2024-09-16T11:08:41.454 INFO:teuthology.orchestra.run.smithi102.stdout:checking clone #5, image image_client.0-clone5 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent6 2024-09-16T11:08:43.215 INFO:teuthology.orchestra.run.smithi102.stdout:compare and write disabled 2024-09-16T11:08:43.216 INFO:teuthology.orchestra.run.smithi102.stdout:921 write 0xa60bdf thru 0xa6e9aa (0xddcc bytes) 2024-09-16T11:08:43.602 INFO:teuthology.orchestra.run.smithi102.stdout:922 trunc from 0x4fb7cfa to 0x1f08a79 2024-09-16T11:08:43.782 INFO:teuthology.orchestra.run.smithi102.stdout:923 read 0x188c88 thru 0x193391 (0xa70a bytes) 2024-09-16T11:08:43.826 INFO:teuthology.orchestra.run.smithi102.stdout:924 writesame 0x518df76 thru 0x519c060 data_size 0x11(0xe0eb bytes) 2024-09-16T11:08:43.827 INFO:teuthology.orchestra.run.smithi102.stdout:925 read 0x3ab075f thru 0x3ab5dd6 (0x5678 bytes) 2024-09-16T11:08:43.828 INFO:teuthology.orchestra.run.smithi102.stdout:compare and write disabled 2024-09-16T11:08:43.828 INFO:teuthology.orchestra.run.smithi102.stdout:928 punch from 0x3d6cb38 to 0x3d74bac, (0x8074 bytes) 2024-09-16T11:08:43.829 INFO:teuthology.orchestra.run.smithi102.stdout:930 write 0x369b652 thru 0x36aa627 (0xefd6 bytes) 2024-09-16T11:08:43.850 INFO:teuthology.orchestra.run.smithi102.stdout:932 read 0x1a915f8 thru 0x1a9af8c (0x9995 bytes) 2024-09-16T11:08:43.856 INFO:teuthology.orchestra.run.smithi102.stdout:933 read 0x604b4a thru 0x60dcbf (0x9176 bytes) 2024-09-16T11:08:43.891 INFO:teuthology.orchestra.run.smithi102.stdout:934 writesame 0x7ceb101 thru 0x7cf6f95 data_size 0x75(0xbe95 bytes) 2024-09-16T11:08:43.892 INFO:teuthology.orchestra.run.smithi102.stdout:935 punch from 0x39838f8 to 0x3990e69, (0xd571 bytes) 2024-09-16T11:08:43.892 INFO:teuthology.orchestra.run.smithi102.stdout:937 read 0x2d570c thru 0x2dc6f7 (0x6fec bytes) 2024-09-16T11:08:43.898 INFO:teuthology.orchestra.run.smithi102.stdout:compare and write disabled 2024-09-16T11:08:43.898 INFO:teuthology.orchestra.run.smithi102.stdout:compare and write disabled 2024-09-16T11:08:43.904 INFO:teuthology.orchestra.run.smithi102.stdout:942 trunc from 0x7cf6f96 to 0x6dee122 2024-09-16T11:08:43.921 INFO:teuthology.orchestra.run.smithi102.stdout:943 punch from 0x3297fa2 to 0x32982f5, (0x353 bytes) 2024-09-16T11:08:43.922 INFO:teuthology.orchestra.run.smithi102.stdout:944 write 0x49840f4 thru 0x4985cc9 (0x1bd6 bytes) 2024-09-16T11:08:43.926 INFO:teuthology.orchestra.run.smithi102.stdout:947 read 0x67390ae thru 0x6746125 (0xd078 bytes) 2024-09-16T11:08:43.935 INFO:teuthology.orchestra.run.smithi102.stdout:948 writesame 0x6a9a59 thru 0x6b8c3f data_size 0x1(0xf1e7 bytes) 2024-09-16T11:08:44.298 INFO:teuthology.orchestra.run.smithi102.stdout:949 write 0x5be2603 thru 0x5be7a96 (0x5494 bytes) 2024-09-16T11:08:44.307 INFO:teuthology.orchestra.run.smithi102.stdout:950 read 0x147e7c9 thru 0x1484d23 (0x655b bytes) 2024-09-16T11:08:44.312 INFO:teuthology.orchestra.run.smithi102.stdout:951 write 0x591af1d thru 0x591beb2 (0xf96 bytes) 2024-09-16T11:08:44.321 INFO:teuthology.orchestra.run.smithi102.stdout:952 writesame 0x66e076e thru 0x66eac42 data_size 0x1(0xa4d5 bytes) 2024-09-16T11:08:44.341 INFO:teuthology.orchestra.run.smithi102.stdout:compare and write disabled 2024-09-16T11:08:44.341 INFO:teuthology.orchestra.run.smithi102.stdout:955 write 0x61438e9 thru 0x6144e55 (0x156d bytes) 2024-09-16T11:08:44.348 INFO:teuthology.orchestra.run.smithi102.stdout:956 writesame 0x43f1de2 thru 0x43f9ff2 data_size 0xb(0x8211 bytes) 2024-09-16T11:08:44.393 INFO:teuthology.orchestra.run.smithi102.stdout:compare and write disabled 2024-09-16T11:08:44.393 INFO:teuthology.orchestra.run.smithi102.stdout:compare and write disabled 2024-09-16T11:08:44.412 INFO:teuthology.orchestra.run.smithi102.stdout:961 trunc from 0x6dee122 to 0x422dbe3 2024-09-16T11:08:44.449 INFO:teuthology.orchestra.run.smithi102.stdout:962 punch from 0x23b5fa2 to 0x23bc982, (0x69e0 bytes) 2024-09-16T11:08:44.480 INFO:teuthology.orchestra.run.smithi102.stdout:964 write 0x67c5efc thru 0x67d1954 (0xba59 bytes) 2024-09-16T11:08:44.499 INFO:teuthology.orchestra.run.smithi102.stdout:compare and write disabled 2024-09-16T11:08:44.508 INFO:teuthology.orchestra.run.smithi102.stdout:969 trunc from 0x67d1955 to 0x7d4e831 2024-09-16T11:08:44.523 INFO:teuthology.orchestra.run.smithi102.stdout:970 punch from 0x3726e27 to 0x3735d0b, (0xeee4 bytes) 2024-09-16T11:08:44.524 INFO:teuthology.orchestra.run.smithi102.stdout:971 punch from 0x125ecfe to 0x1265354, (0x6656 bytes) 2024-09-16T11:08:44.525 INFO:teuthology.orchestra.run.smithi102.stdout:972 write 0x337735d thru 0x3385b6f (0xe813 bytes) 2024-09-16T11:08:44.566 INFO:teuthology.orchestra.run.smithi102.stdout:973 trunc from 0x7d4e831 to 0x51f1202 2024-09-16T11:08:44.617 INFO:teuthology.orchestra.run.smithi102.stdout:974 write 0x691e632 thru 0x692acd8 (0xc6a7 bytes) 2024-09-16T11:08:44.637 INFO:teuthology.orchestra.run.smithi102.stdout:976 write 0xd1d72a thru 0xd29a95 (0xc36c bytes) 2024-09-16T11:08:44.946 INFO:teuthology.orchestra.run.smithi102.stdout:977 read 0x52e5592 thru 0x52e9b95 (0x4604 bytes) 2024-09-16T11:08:44.947 INFO:teuthology.orchestra.run.smithi102.stdout:978 writesame 0x56ef920 thru 0x56f22c7 data_size 0x3e(0x29a8 bytes) 2024-09-16T11:08:44.961 INFO:teuthology.orchestra.run.smithi102.stdout:979 read 0x3c4e48a thru 0x3c51f85 (0x3afc bytes) 2024-09-16T11:08:44.961 INFO:teuthology.orchestra.run.smithi102.stdout:980 writesame 0x1993cd thru 0x19b57b data_size 0x1(0x21af bytes) 2024-09-16T11:08:45.060 INFO:teuthology.orchestra.run.smithi102.stdout:981 write 0x64c517 thru 0x65ada9 (0xe893 bytes) 2024-09-16T11:08:45.082 INFO:teuthology.orchestra.run.smithi102.stdout:982 trunc from 0x692acd9 to 0x4c7fa0a 2024-09-16T11:08:45.110 INFO:teuthology.orchestra.run.smithi102.stdout:984 punch from 0x17b5fa to 0x17f3f7, (0x3dfd bytes) 2024-09-16T11:08:45.110 INFO:teuthology.orchestra.run.smithi102.stdout:985 read 0xb3829 thru 0xc31d3 (0xf9ab bytes) 2024-09-16T11:08:45.119 INFO:teuthology.orchestra.run.smithi102.stdout:986 punch from 0x351a4b6 to 0x352381d, (0x9367 bytes) 2024-09-16T11:08:45.119 INFO:teuthology.orchestra.run.smithi102.stdout:988 read 0x1892a75 thru 0x189acba (0x8246 bytes) 2024-09-16T11:08:45.146 INFO:teuthology.orchestra.run.smithi102.stdout:990 write 0x5ae067f thru 0x5aec41d (0xbd9f bytes) 2024-09-16T11:08:45.163 INFO:teuthology.orchestra.run.smithi102.stdout:991 write 0x3258c36 thru 0x3260429 (0x77f4 bytes) 2024-09-16T11:08:45.169 INFO:teuthology.orchestra.run.smithi102.stdout:992 read 0x2b539ed thru 0x2b5c6c2 (0x8cd6 bytes) 2024-09-16T11:08:45.170 INFO:teuthology.orchestra.run.smithi102.stdout:994 writesame 0x277dfde thru 0x278b669 data_size 0x17(0xd68c bytes) 2024-09-16T11:08:45.215 INFO:teuthology.orchestra.run.smithi102.stdout:995 punch from 0x2307032 to 0x230f47e, (0x844c bytes) 2024-09-16T11:08:45.249 INFO:teuthology.orchestra.run.smithi102.stdout:997 writesame 0x7770d69 thru 0x777f74f data_size 0x1(0xe9e7 bytes) 2024-09-16T11:08:45.249 INFO:teuthology.orchestra.run.smithi102.stdout:998 punch from 0x72b999e to 0x72c2eb3, (0x9515 bytes) 2024-09-16T11:08:45.250 INFO:teuthology.orchestra.run.smithi102.stdout:999 write 0x3372286 thru 0x3378bd6 (0x6951 bytes) 2024-09-16T11:08:45.330 INFO:teuthology.orchestra.run.smithi102.stdout:checking clone #6, image image_client.0-clone6 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent7 2024-09-16T11:08:54.219 INFO:teuthology.orchestra.run.smithi102.stdout:All operations completed A-OK! 2024-09-16T11:08:54.486 DEBUG:teuthology.parallel:result is None 2024-09-16T11:08:54.486 DEBUG:teuthology.run_tasks:Unwinding manager rbd_fsx 2024-09-16T11:08:54.507 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-09-16T11:08:54.544 INFO:tasks.ceph.ceph_manager.ceph:Canceling any pending splits or merges... 2024-09-16T11:08:54.544 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-16T11:08:54.804 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-16T11:08:54.804 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":35,"fsid":"79f34890-6ac0-492b-826f-b29cf1045a98","created":"2024-09-16T11:07:27.825113+0000","modified":"2024-09-16T11:08:53.878423+0000","last_up_change":"2024-09-16T11:07:37.411085+0000","last_in_change":"2024-09-16T11:07:28.722914+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T11:07:37.676106+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"11","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-16T11:07:42.972213+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":15,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"pool_client.0","create_time":"2024-09-16T11:08:06.142273+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":17,"snap_epoch":35,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"73775a88-4914-46f2-a066-b4f012c226c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6800","nonce":32395},{"type":"v1","addr":"172.21.15.102:6801","nonce":32395}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6802","nonce":32395},{"type":"v1","addr":"172.21.15.102:6803","nonce":32395}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6806","nonce":32395},{"type":"v1","addr":"172.21.15.102:6807","nonce":32395}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6804","nonce":32395},{"type":"v1","addr":"172.21.15.102:6805","nonce":32395}]},"public_addr":"172.21.15.102:6801/32395","cluster_addr":"172.21.15.102:6803/32395","heartbeat_back_addr":"172.21.15.102:6807/32395","heartbeat_front_addr":"172.21.15.102:6805/32395","state":["exists","up"]},{"osd":1,"uuid":"566bbdc4-ed6a-4580-a04b-7d4fa97fcc81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6816","nonce":32397},{"type":"v1","addr":"172.21.15.102:6817","nonce":32397}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6818","nonce":32397},{"type":"v1","addr":"172.21.15.102:6819","nonce":32397}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6822","nonce":32397},{"type":"v1","addr":"172.21.15.102:6823","nonce":32397}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6820","nonce":32397},{"type":"v1","addr":"172.21.15.102:6821","nonce":32397}]},"public_addr":"172.21.15.102:6817/32397","cluster_addr":"172.21.15.102:6819/32397","heartbeat_back_addr":"172.21.15.102:6823/32397","heartbeat_front_addr":"172.21.15.102:6821/32397","state":["exists","up"]},{"osd":2,"uuid":"4aaa7267-412c-4e0f-b6f9-dd70e45cdc8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6808","nonce":32396},{"type":"v1","addr":"172.21.15.102:6809","nonce":32396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6810","nonce":32396},{"type":"v1","addr":"172.21.15.102:6811","nonce":32396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6814","nonce":32396},{"type":"v1","addr":"172.21.15.102:6815","nonce":32396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6812","nonce":32396},{"type":"v1","addr":"172.21.15.102:6813","nonce":32396}]},"public_addr":"172.21.15.102:6809/32396","cluster_addr":"172.21.15.102:6811/32396","heartbeat_back_addr":"172.21.15.102:6815/32396","heartbeat_front_addr":"172.21.15.102:6813/32396","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T11:07:33.892765+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T11:07:33.888914+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T11:07:33.917416+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":3,"snaps":[{"begin":4,"length":7}]}],"new_removed_snaps":[{"pool":3,"snaps":[{"begin":4,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T11:08:54.816 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-16T11:08:54.817 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T11:08:55.075 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-16T11:08:55.076 INFO:teuthology.orchestra.run.smithi102.stderr:dumped all 2024-09-16T11:08:55.086 INFO:teuthology.orchestra.run.smithi102.stdout:{"pg_ready":true,"pg_map":{"version":71,"stamp":"2024-09-16T11:08:53.880660+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":38133882,"num_objects":63,"num_object_clones":19,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63,"num_whiteouts":7,"num_read":15710,"num_read_kb":48777,"num_write":3030,"num_write_kb":29004,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2401,"ondisk_log_size":2401,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":917432,"kb_used_data":4560,"kb_used_omap":4,"kb_used_meta":912827,"kb_avail":282198088,"statfs":{"total":289910292480,"available":288970842112,"internally_reserved":0,"allocated":4669440,"data_stored":4850886,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":5112,"internal_metadata":934734856},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-161246497,"num_objects":-114,"num_object_clones":-24,"num_object_copies":-228,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-114,"num_whiteouts":-19,"num_read":1251,"num_read_kb":1071,"num_write":242,"num_write_kb":63,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.021852"},"pg_stats":[{"pgid":"3.1f","version":"33'33","reported_seq":199,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-16T11:08:51.888152+0000","last_change":"2024-09-16T11:08:51.888351+0000","last_active":"2024-09-16T11:08:51.888152+0000","last_peered":"2024-09-16T11:08:51.888152+0000","last_clean":"2024-09-16T11:08:51.888152+0000","last_became_active":"2024-09-16T11:08:07.859102+0000","last_became_peered":"2024-09-16T11:08:07.859102+0000","last_unstale":"2024-09-16T11:08:51.888152+0000","last_undegraded":"2024-09-16T11:08:51.888152+0000","last_fullsized":"2024-09-16T11:08:51.888152+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:08:06.827173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:08:06.827173+0000","last_clean_scrub_stamp":"2024-09-16T11:08:06.827173+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":0,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:21:53.809689+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.008213639,"stat_sum":{"num_bytes":2813157,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":127,"num_read_kb":1006,"num_write":34,"num_write_kb":861,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"6","length":"5"}]},{"pgid":"3.1e","version":"33'155","reported_seq":741,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-16T11:08:52.349329+0000","last_change":"2024-09-16T11:08:51.877347+0000","last_active":"2024-09-16T11:08:52.349329+0000","last_peered":"2024-09-16T11:08:52.349329+0000","last_clean":"2024-09-16T11:08:52.349329+0000","last_became_active":"2024-09-16T11:08:07.856092+0000","last_became_peered":"2024-09-16T11:08:07.856092+0000","last_unstale":"2024-09-16T11:08:52.349329+0000","last_undegraded":"2024-09-16T11:08:52.349329+0000","last_fullsized":"2024-09-16T11:08:52.349329+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:08:06.827173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:08:06.827173+0000","last_clean_scrub_stamp":"2024-09-16T11:08:06.827173+0000","objects_scrubbed":0,"log_size":155,"log_dups_size":0,"ondisk_log_size":155,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:02:44.147643+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0023786520000000002,"stat_sum":{"num_bytes":37,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1807,"num_read_kb":2532,"num_write":196,"num_write_kb":424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"6","length":"5"}]},{"pgid":"3.1d","version":"28'275","reported_seq":332,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-16T11:08:51.877757+0000","last_change":"2024-09-16T11:08:51.878043+0000","last_active":"2024-09-16T11:08:51.877757+0000","last_peered":"2024-09-16T11:08:51.877757+0000","last_clean":"2024-09-16T11:08:51.877757+0000","last_became_active":"2024-09-16T11:08:07.847513+0000","last_became_peered":"2024-09-16T11:08:07.847513+0000","last_unstale":"2024-09-16T11:08:51.877757+0000","last_undegraded":"2024-09-16T11:08:51.877757+0000","last_fullsized":"2024-09-16T11:08:51.877757+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:08:06.827173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:08:06.827173+0000","last_clean_scrub_stamp":"2024-09-16T11:08:06.827173+0000","objects_scrubbed":0,"log_size":275,"log_dups_size":0,"ondisk_log_size":275,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:51:39.065833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.003055496,"stat_sum":{"num_bytes":42,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1234,"num_read_kb":1012,"num_write":445,"num_write_kb":597,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"6","length":"5"}]},{"pgid":"3.1c","version":"33'33","reported_seq":213,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-16T11:08:52.109073+0000","last_change":"2024-09-16T11:08:51.882421+0000","last_active":"2024-09-16T11:08:52.109073+0000","last_peered":"2024-09-16T11:08:52.109073+0000","last_clean":"2024-09-16T11:08:52.109073+0000","last_became_active":"2024-09-16T11:08:07.855430+0000","last_became_peered":"2024-09-16T11:08:07.855430+0000","last_unstale":"2024-09-16T11:08:52.109073+0000","last_undegraded":"2024-09-16T11:08:52.109073+0000","last_fullsized":"2024-09-16T11:08:52.109073+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:08:06.827173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:08:06.827173+0000","last_clean_scrub_stamp":"2024-09-16T11:08:06.827173+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":0,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:28:07.715262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0048504100000000003,"stat_sum":{"num_bytes":1869110,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":167,"num_read_kb":250,"num_write":42,"num_write_kb":566,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"6","length":"5"}]},{"pgid":"3.1b","version":"33'160","reported_seq":840,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-16T11:08:51.888964+0000","last_change":"2024-09-16T11:08:51.889166+0000","last_active":"2024-09-16T11:08:51.888964+0000","last_peered":"2024-09-16T11:08:51.888964+0000","last_clean":"2024-09-16T11:08:51.888964+0000","last_became_active":"2024-09-16T11:08:07.856827+0000","last_became_peered":"2024-09-16T11:08:07.856827+0000","last_unstale":"2024-09-16T11:08:51.888964+0000","last_undegraded":"2024-09-16T11:08:51.888964+0000","last_fullsized":"2024-09-16T11:08:51.888964+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:08:06.827173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:08:06.827173+0000","last_clean_scrub_stamp":"2024-09-16T11:08:06.827173+0000","objects_scrubbed":0,"log_size":160,"log_dups_size":0,"ondisk_log_size":160,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:47:27.295935+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0088276499999999994,"stat_sum":{"num_bytes":590386,"num_objects":4,"num_object_clones":2,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":1,"num_read":1407,"num_read_kb":2244,"num_write":139,"num_write_kb":887,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"6","length":"5"}]},{"pgid":"3.1a","version":"28'13","reported_seq":374,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-16T11:08:52.316726+0000","last_change":"2024-09-16T11:08:51.887798+0000","last_active":"2024-09-16T11:08:52.316726+0000","last_peered":"2024-09-16T11:08:52.316726+0000","last_clean":"2024-09-16T11:08:52.316726+0000","last_became_active":"2024-09-16T11:08:07.853372+0000","last_became_peered":"2024-09-16T11:08:07.853372+0000","last_unstale":"2024-09-16T11:08:52.316726+0000","last_undegraded":"2024-09-16T11:08:52.316726+0000","last_fullsized":"2024-09-16T11:08:52.316726+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:08:06.827173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:08:06.827173+0000","last_clean_scrub_stamp":"2024-09-16T11:08:06.827173+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:21:53.809689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0089025300000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":315,"num_read_kb":333,"num_write":13,"num_write_kb":81,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"6","length":"5"}]},{"pgid":"3.19","version":"32'39","reported_seq":280,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-16T11:08:51.881126+0000","last_change":"2024-09-16T11:08:51.881384+0000","last_active":"2024-09-16T11:08:51.881126+0000","last_peered":"2024-09-16T11:08:51.881126+0000","last_clean":"2024-09-16T11:08:51.881126+0000","last_became_active":"2024-09-16T11:08:07.852874+0000","last_became_peered":"2024-09-16T11:08:07.852874+0000","last_unstale":"2024-09-16T11:08:51.881126+0000","last_undegraded":"2024-09-16T11:08:51.881126+0000","last_fullsized":"2024-09-16T11:08:51.881126+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:08:06.827173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:08:06.827173+0000","last_clean_scrub_stamp":"2024-09-16T11:08:06.827173+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:47:27.295935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0042733110000000001,"stat_sum":{"num_bytes":6202271,"num_objects":6,"num_object_clones":2,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":214,"num_read_kb":667,"num_write":42,"num_write_kb":746,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"6","length":"5"}]},{"pgid":"3.18","version":"28'17","reported_seq":66,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-16T11:08:51.874889+0000","last_change":"2024-09-16T11:08:51.875249+0000","last_active":"2024-09-16T11:08:51.874889+0000","last_peered":"2024-09-16T11:08:51.874889+0000","last_clean":"2024-09-16T11:08:51.874889+0000","last_became_active":"2024-09-16T11:08:07.848434+0000","last_became_peered":"2024-09-16T11:08:07.848434+0000","last_unstale":"2024-09-16T11:08:51.874889+0000","last_undegraded":"2024-09-16T11:08:51.874889+0000","last_fullsized":"2024-09-16T11:08:51.874889+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:08:06.827173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:08:06.827173+0000","last_clean_scrub_stamp":"2024-09-16T11:08:06.827173+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:47:04.478053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055608000000000003,"stat_sum":{"num_bytes":2342210,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":17,"num_write_kb":60,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"6","length":"5"}]},{"pgid":"3.17","version":"31'25","reported_seq":99,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-16T11:08:51.884747+0000","last_change":"2024-09-16T11:08:51.885023+0000","last_active":"2024-09-16T11:08:51.884747+0000","last_peered":"2024-09-16T11:08:51.884747+0000","last_clean":"2024-09-16T11:08:51.884747+0000","last_became_active":"2024-09-16T11:08:07.859097+0000","last_became_peered":"2024-09-16T11:08:07.859097+0000","last_unstale":"2024-09-16T11:08:51.884747+0000","last_undegraded":"2024-09-16T11:08:51.884747+0000","last_fullsized":"2024-09-16T11:08:51.884747+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:08:06.827173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:08:06.827173+0000","last_clean_scrub_stamp":"2024-09-16T11:08:06.827173+0000","objects_scrubbed":0,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:33:25.601273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0048877390000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":30,"num_read_kb":66,"num_write":24,"num_write_kb":325,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"6","length":"5"}]},{"pgid":"3.16","version":"32'25","reported_seq":128,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-16T11:08:51.878256+0000","last_change":"2024-09-16T11:08:51.878537+0000","last_active":"2024-09-16T11:08:51.878256+0000","last_peered":"2024-09-16T11:08:51.878256+0000","last_clean":"2024-09-16T11:08:51.878256+0000","last_became_active":"2024-09-16T11:08:07.856007+0000","last_became_peered":"2024-09-16T11:08:07.856007+0000","last_unstale":"2024-09-16T11:08:51.878256+0000","last_undegraded":"2024-09-16T11:08:51.878256+0000","last_fullsized":"2024-09-16T11:08:51.878256+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:08:06.827173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:08:06.827173+0000","last_clean_scrub_stamp":"2024-09-16T11:08:06.827173+0000","objects_scrubbed":0,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:18:30.121459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.003443495,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":66,"num_read_kb":1713,"num_write":38,"num_write_kb":1539,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"6","length":"5"}]},{"pgid":"3.15","version":"33'31","reported_seq":160,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-16T11:08:51.884698+0000","last_change":"2024-09-16T11:08:51.884698+0000","last_active":"2024-09-16T11:08:51.884698+0000","last_peered":"2024-09-16T11:08:51.884698+0000","last_clean":"2024-09-16T11:08:51.884698+0000","last_became_active":"2024-09-16T11:08:07.858375+0000","last_became_peered":"2024-09-16T11:08:07.858375+0000","last_unstale":"2024-09-16T11:08:51.884698+0000","last_undegraded":"2024-09-16T11:08:51.884698+0000","last_fullsized":"2024-09-16T11:08:51.884698+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:08:06.827173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:08:06.827173+0000","last_clean_scrub_stamp":"2024-09-16T11:08:06.827173+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:46:06.267425+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.0047413910000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":92,"num_read_kb":770,"num_write":36,"num_write_kb":1093,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"6","length":"5"}]},{"pgid":"3.14","version":"32'137","reported_seq":289,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-16T11:08:51.882715+0000","last_change":"2024-09-16T11:08:51.883437+0000","last_active":"2024-09-16T11:08:51.882715+0000","last_peered":"2024-09-16T11:08:51.882715+0000","last_clean":"2024-09-16T11:08:51.882715+0000","last_became_active":"2024-09-16T11:08:07.853189+0000","last_became_peered":"2024-09-16T11:08:07.853189+0000","last_unstale":"2024-09-16T11:08:51.882715+0000","last_undegraded":"2024-09-16T11:08:51.882715+0000","last_fullsized":"2024-09-16T11:08:51.882715+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:08:06.827173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:08:06.827173+0000","last_clean_scrub_stamp":"2024-09-16T11:08:06.827173+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:30:04.184361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0057208049999999998,"stat_sum":{"num_bytes":1600291,"num_objects":3,"num_object_clones":1,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":523,"num_read_kb":930,"num_write":187,"num_write_kb":943,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"6","length":"5"}]},{"pgid":"2.7","version":"0'0","reported_seq":33,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-16T11:08:50.865069+0000","last_change":"2024-09-16T11:07:46.454524+0000","last_active":"2024-09-16T11:08:50.865069+0000","last_peered":"2024-09-16T11:08:50.865069+0000","last_clean":"2024-09-16T11:08:50.865069+0000","last_became_active":"2024-09-16T11:07:44.446646+0000","last_became_peered":"2024-09-16T11:07:44.446646+0000","last_unstale":"2024-09-16T11:08:50.865069+0000","last_undegraded":"2024-09-16T11:08:50.865069+0000","last_fullsized":"2024-09-16T11:08:50.865069+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:07:43.427920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:07:43.427920+0000","last_clean_scrub_stamp":"2024-09-16T11:07:43.427920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:34:38.920978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000337344,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.6","version":"33'80","reported_seq":135,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-16T11:08:51.886576+0000","last_change":"2024-09-16T11:08:51.886576+0000","last_active":"2024-09-16T11:08:51.886576+0000","last_peered":"2024-09-16T11:08:51.886576+0000","last_clean":"2024-09-16T11:08:51.886576+0000","last_became_active":"2024-09-16T11:08:07.858208+0000","last_became_peered":"2024-09-16T11:08:07.858208+0000","last_unstale":"2024-09-16T11:08:51.886576+0000","last_undegraded":"2024-09-16T11:08:51.886576+0000","last_fullsized":"2024-09-16T11:08:51.886576+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:08:06.827173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:08:06.827173+0000","last_clean_scrub_stamp":"2024-09-16T11:08:06.827173+0000","objects_scrubbed":0,"log_size":80,"log_dups_size":0,"ondisk_log_size":80,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:20:16.980280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0063318970000000004,"stat_sum":{"num_bytes":3631436,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":259,"num_read_kb":213,"num_write":113,"num_write_kb":675,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"6","length":"5"}]},{"pgid":"2.6","version":"0'0","reported_seq":33,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-16T11:08:50.864872+0000","last_change":"2024-09-16T11:07:46.454822+0000","last_active":"2024-09-16T11:08:50.864872+0000","last_peered":"2024-09-16T11:08:50.864872+0000","last_clean":"2024-09-16T11:08:50.864872+0000","last_became_active":"2024-09-16T11:07:44.446406+0000","last_became_peered":"2024-09-16T11:07:44.446406+0000","last_unstale":"2024-09-16T11:08:50.864872+0000","last_undegraded":"2024-09-16T11:08:50.864872+0000","last_fullsized":"2024-09-16T11:08:50.864872+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:07:43.427920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:07:43.427920+0000","last_clean_scrub_stamp":"2024-09-16T11:07:43.427920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:12:45.650225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062301399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"31'41","reported_seq":314,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-16T11:08:51.878097+0000","last_change":"2024-09-16T11:08:51.878390+0000","last_active":"2024-09-16T11:08:51.878097+0000","last_peered":"2024-09-16T11:08:51.878097+0000","last_clean":"2024-09-16T11:08:51.878097+0000","last_became_active":"2024-09-16T11:08:07.856393+0000","last_became_peered":"2024-09-16T11:08:07.856393+0000","last_unstale":"2024-09-16T11:08:51.878097+0000","last_undegraded":"2024-09-16T11:08:51.878097+0000","last_fullsized":"2024-09-16T11:08:51.878097+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:08:06.827173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:08:06.827173+0000","last_clean_scrub_stamp":"2024-09-16T11:08:06.827173+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:11:35.974778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.003312602,"stat_sum":{"num_bytes":4131275,"num_objects":3,"num_object_clones":1,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":1,"num_read":241,"num_read_kb":2845,"num_write":54,"num_write_kb":2044,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"6","length":"5"}]},{"pgid":"2.5","version":"0'0","reported_seq":33,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-16T11:08:50.864449+0000","last_change":"2024-09-16T11:07:46.455055+0000","last_active":"2024-09-16T11:08:50.864449+0000","last_peered":"2024-09-16T11:08:50.864449+0000","last_clean":"2024-09-16T11:08:50.864449+0000","last_became_active":"2024-09-16T11:07:44.446498+0000","last_became_peered":"2024-09-16T11:07:44.446498+0000","last_unstale":"2024-09-16T11:08:50.864449+0000","last_undegraded":"2024-09-16T11:08:50.864449+0000","last_fullsized":"2024-09-16T11:08:50.864449+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:07:43.427920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:07:43.427920+0000","last_clean_scrub_stamp":"2024-09-16T11:07:43.427920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:40:08.726582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00084251400000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"32'67","reported_seq":266,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-16T11:08:51.883487+0000","last_change":"2024-09-16T11:08:51.883852+0000","last_active":"2024-09-16T11:08:51.883487+0000","last_peered":"2024-09-16T11:08:51.883487+0000","last_clean":"2024-09-16T11:08:51.883487+0000","last_became_active":"2024-09-16T11:08:07.853470+0000","last_became_peered":"2024-09-16T11:08:07.853470+0000","last_unstale":"2024-09-16T11:08:51.883487+0000","last_undegraded":"2024-09-16T11:08:51.883487+0000","last_fullsized":"2024-09-16T11:08:51.883487+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:08:06.827173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:08:06.827173+0000","last_clean_scrub_stamp":"2024-09-16T11:08:06.827173+0000","objects_scrubbed":0,"log_size":67,"log_dups_size":0,"ondisk_log_size":67,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:46:06.267425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0059983930000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":360,"num_read_kb":1468,"num_write":100,"num_write_kb":888,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"6","length":"5"}]},{"pgid":"2.4","version":"0'0","reported_seq":33,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-16T11:08:50.864582+0000","last_change":"2024-09-16T11:07:46.454774+0000","last_active":"2024-09-16T11:08:50.864582+0000","last_peered":"2024-09-16T11:08:50.864582+0000","last_clean":"2024-09-16T11:08:50.864582+0000","last_became_active":"2024-09-16T11:07:44.446413+0000","last_became_peered":"2024-09-16T11:07:44.446413+0000","last_unstale":"2024-09-16T11:08:50.864582+0000","last_undegraded":"2024-09-16T11:08:50.864582+0000","last_fullsized":"2024-09-16T11:08:50.864582+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:07:43.427920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:07:43.427920+0000","last_clean_scrub_stamp":"2024-09-16T11:07:43.427920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:53:07.204424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059042800000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"32'69","reported_seq":469,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-16T11:08:51.878615+0000","last_change":"2024-09-16T11:08:51.879403+0000","last_active":"2024-09-16T11:08:51.878615+0000","last_peered":"2024-09-16T11:08:51.878615+0000","last_clean":"2024-09-16T11:08:51.878615+0000","last_became_active":"2024-09-16T11:08:07.855280+0000","last_became_peered":"2024-09-16T11:08:07.855280+0000","last_unstale":"2024-09-16T11:08:51.878615+0000","last_undegraded":"2024-09-16T11:08:51.878615+0000","last_fullsized":"2024-09-16T11:08:51.878615+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:08:06.827173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:08:06.827173+0000","last_clean_scrub_stamp":"2024-09-16T11:08:06.827173+0000","objects_scrubbed":0,"log_size":69,"log_dups_size":0,"ondisk_log_size":69,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:03:42.315998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0042484180000000003,"stat_sum":{"num_bytes":786432,"num_objects":6,"num_object_clones":3,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":396,"num_read_kb":2349,"num_write":76,"num_write_kb":1069,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"6","length":"5"}]},{"pgid":"3.3","version":"33'78","reported_seq":247,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-16T11:08:52.210563+0000","last_change":"2024-09-16T11:08:51.886868+0000","last_active":"2024-09-16T11:08:52.210563+0000","last_peered":"2024-09-16T11:08:52.210563+0000","last_clean":"2024-09-16T11:08:52.210563+0000","last_became_active":"2024-09-16T11:08:07.847550+0000","last_became_peered":"2024-09-16T11:08:07.847550+0000","last_unstale":"2024-09-16T11:08:52.210563+0000","last_undegraded":"2024-09-16T11:08:52.210563+0000","last_fullsized":"2024-09-16T11:08:52.210563+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:08:06.827173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:08:06.827173+0000","last_clean_scrub_stamp":"2024-09-16T11:08:06.827173+0000","objects_scrubbed":0,"log_size":78,"log_dups_size":0,"ondisk_log_size":78,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:25:26.863744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0065329400000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":152,"num_read_kb":956,"num_write":108,"num_write_kb":2573,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"6","length":"5"}]},{"pgid":"2.2","version":"15'2","reported_seq":34,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-16T11:08:50.869072+0000","last_change":"2024-09-16T11:07:46.448909+0000","last_active":"2024-09-16T11:08:50.869072+0000","last_peered":"2024-09-16T11:08:50.869072+0000","last_clean":"2024-09-16T11:08:50.869072+0000","last_became_active":"2024-09-16T11:07:44.449069+0000","last_became_peered":"2024-09-16T11:07:44.449069+0000","last_unstale":"2024-09-16T11:08:50.869072+0000","last_undegraded":"2024-09-16T11:08:50.869072+0000","last_fullsized":"2024-09-16T11:08:50.869072+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:07:43.427920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:07:43.427920+0000","last_clean_scrub_stamp":"2024-09-16T11:07:43.427920+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:13:44.990400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00029305100000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.0","version":"33'46","reported_seq":203,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-16T11:08:52.184769+0000","last_change":"2024-09-16T11:08:51.881057+0000","last_active":"2024-09-16T11:08:52.184769+0000","last_peered":"2024-09-16T11:08:52.184769+0000","last_clean":"2024-09-16T11:08:52.184769+0000","last_became_active":"2024-09-16T11:08:07.855258+0000","last_became_peered":"2024-09-16T11:08:07.855258+0000","last_unstale":"2024-09-16T11:08:52.184769+0000","last_undegraded":"2024-09-16T11:08:52.184769+0000","last_fullsized":"2024-09-16T11:08:52.184769+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:08:06.827173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:08:06.827173+0000","last_clean_scrub_stamp":"2024-09-16T11:08:06.827173+0000","objects_scrubbed":0,"log_size":46,"log_dups_size":0,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:26:07.701673+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.0040037609999999998,"stat_sum":{"num_bytes":42,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":139,"num_read_kb":1672,"num_write":52,"num_write_kb":1263,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"6","length":"5"}]},{"pgid":"2.1","version":"0'0","reported_seq":32,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-16T11:08:50.861810+0000","last_change":"2024-09-16T11:07:46.720289+0000","last_active":"2024-09-16T11:08:50.861810+0000","last_peered":"2024-09-16T11:08:50.861810+0000","last_clean":"2024-09-16T11:08:50.861810+0000","last_became_active":"2024-09-16T11:07:44.447533+0000","last_became_peered":"2024-09-16T11:07:44.447533+0000","last_unstale":"2024-09-16T11:08:50.861810+0000","last_undegraded":"2024-09-16T11:08:50.861810+0000","last_fullsized":"2024-09-16T11:08:50.861810+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:07:43.427920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:07:43.427920+0000","last_clean_scrub_stamp":"2024-09-16T11:07:43.427920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:13:44.990400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000230378,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"33'30","reported_seq":160,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-16T11:08:51.882164+0000","last_change":"2024-09-16T11:08:51.883251+0000","last_active":"2024-09-16T11:08:51.882164+0000","last_peered":"2024-09-16T11:08:51.882164+0000","last_clean":"2024-09-16T11:08:51.882164+0000","last_became_active":"2024-09-16T11:08:07.847955+0000","last_became_peered":"2024-09-16T11:08:07.847955+0000","last_unstale":"2024-09-16T11:08:51.882164+0000","last_undegraded":"2024-09-16T11:08:51.882164+0000","last_fullsized":"2024-09-16T11:08:51.882164+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:08:06.827173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:08:06.827173+0000","last_clean_scrub_stamp":"2024-09-16T11:08:06.827173+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:02:44.147643+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.003530334,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":87,"num_read_kb":1164,"num_write":31,"num_write_kb":740,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"6","length":"5"}]},{"pgid":"2.0","version":"0'0","reported_seq":32,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-16T11:08:50.861284+0000","last_change":"2024-09-16T11:07:46.720264+0000","last_active":"2024-09-16T11:08:50.861284+0000","last_peered":"2024-09-16T11:08:50.861284+0000","last_clean":"2024-09-16T11:08:50.861284+0000","last_became_active":"2024-09-16T11:07:44.448153+0000","last_became_peered":"2024-09-16T11:07:44.448153+0000","last_unstale":"2024-09-16T11:08:50.861284+0000","last_undegraded":"2024-09-16T11:08:50.861284+0000","last_fullsized":"2024-09-16T11:08:50.861284+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:07:43.427920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:07:43.427920+0000","last_clean_scrub_stamp":"2024-09-16T11:07:43.427920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:09:42.536129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00024792500000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"33'40","reported_seq":266,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-16T11:08:52.254010+0000","last_change":"2024-09-16T11:08:51.889257+0000","last_active":"2024-09-16T11:08:52.254010+0000","last_peered":"2024-09-16T11:08:52.254010+0000","last_clean":"2024-09-16T11:08:52.254010+0000","last_became_active":"2024-09-16T11:08:07.853307+0000","last_became_peered":"2024-09-16T11:08:07.853307+0000","last_unstale":"2024-09-16T11:08:52.254010+0000","last_undegraded":"2024-09-16T11:08:52.254010+0000","last_fullsized":"2024-09-16T11:08:52.254010+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:08:06.827173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:08:06.827173+0000","last_clean_scrub_stamp":"2024-09-16T11:08:06.827173+0000","objects_scrubbed":0,"log_size":40,"log_dups_size":0,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:33:25.601273+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.010809077,"stat_sum":{"num_bytes":1243751,"num_objects":5,"num_object_clones":2,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":1,"num_read":203,"num_read_kb":1989,"num_write":37,"num_write_kb":217,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"6","length":"5"}]},{"pgid":"2.3","version":"13'1","reported_seq":34,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-16T11:08:50.865131+0000","last_change":"2024-09-16T11:07:46.454516+0000","last_active":"2024-09-16T11:08:50.865131+0000","last_peered":"2024-09-16T11:08:50.865131+0000","last_clean":"2024-09-16T11:08:50.865131+0000","last_became_active":"2024-09-16T11:07:44.446367+0000","last_became_peered":"2024-09-16T11:07:44.446367+0000","last_unstale":"2024-09-16T11:08:50.865131+0000","last_undegraded":"2024-09-16T11:08:50.865131+0000","last_fullsized":"2024-09-16T11:08:50.865131+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:07:43.427920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:07:43.427920+0000","last_clean_scrub_stamp":"2024-09-16T11:07:43.427920+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:06:21.202391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00038771200000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"10'52","reported_seq":98,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-16T11:08:50.864840+0000","last_change":"2024-09-16T11:07:39.420884+0000","last_active":"2024-09-16T11:08:50.864840+0000","last_peered":"2024-09-16T11:08:50.864840+0000","last_clean":"2024-09-16T11:08:50.864840+0000","last_became_active":"2024-09-16T11:07:39.420564+0000","last_became_peered":"2024-09-16T11:07:39.420564+0000","last_unstale":"2024-09-16T11:08:50.864840+0000","last_undegraded":"2024-09-16T11:08:50.864840+0000","last_fullsized":"2024-09-16T11:08:50.864840+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:07:38.411971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:07:38.411971+0000","last_clean_scrub_stamp":"2024-09-16T11:07:38.411971+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:52:50.808244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":56,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"33'41","reported_seq":448,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-16T11:08:51.877660+0000","last_change":"2024-09-16T11:08:51.877660+0000","last_active":"2024-09-16T11:08:51.877660+0000","last_peered":"2024-09-16T11:08:51.877660+0000","last_clean":"2024-09-16T11:08:51.877660+0000","last_became_active":"2024-09-16T11:08:07.857280+0000","last_became_peered":"2024-09-16T11:08:07.857280+0000","last_unstale":"2024-09-16T11:08:51.877660+0000","last_undegraded":"2024-09-16T11:08:51.877660+0000","last_fullsized":"2024-09-16T11:08:51.877660+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:08:06.827173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:08:06.827173+0000","last_clean_scrub_stamp":"2024-09-16T11:08:06.827173+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:25:26.863744+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.0028720669999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":381,"num_read_kb":3183,"num_write":40,"num_write_kb":936,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"6","length":"5"}]},{"pgid":"3.9","version":"28'17","reported_seq":65,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-16T11:08:51.879719+0000","last_change":"2024-09-16T11:08:51.880725+0000","last_active":"2024-09-16T11:08:51.879719+0000","last_peered":"2024-09-16T11:08:51.879719+0000","last_clean":"2024-09-16T11:08:51.879719+0000","last_became_active":"2024-09-16T11:08:07.850403+0000","last_became_peered":"2024-09-16T11:08:07.850403+0000","last_unstale":"2024-09-16T11:08:51.879719+0000","last_undegraded":"2024-09-16T11:08:51.879719+0000","last_fullsized":"2024-09-16T11:08:51.879719+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:08:06.827173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:08:06.827173+0000","last_clean_scrub_stamp":"2024-09-16T11:08:06.827173+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:32:06.761762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011094919999999999,"stat_sum":{"num_bytes":3063089,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":3,"num_read_kb":55,"num_write":20,"num_write_kb":629,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"6","length":"5"}]},{"pgid":"3.a","version":"33'48","reported_seq":361,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-16T11:08:51.886253+0000","last_change":"2024-09-16T11:08:51.886253+0000","last_active":"2024-09-16T11:08:51.886253+0000","last_peered":"2024-09-16T11:08:51.886253+0000","last_clean":"2024-09-16T11:08:51.886253+0000","last_became_active":"2024-09-16T11:08:07.857407+0000","last_became_peered":"2024-09-16T11:08:07.857407+0000","last_unstale":"2024-09-16T11:08:51.886253+0000","last_undegraded":"2024-09-16T11:08:51.886253+0000","last_fullsized":"2024-09-16T11:08:51.886253+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:08:06.827173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:08:06.827173+0000","last_clean_scrub_stamp":"2024-09-16T11:08:06.827173+0000","objects_scrubbed":0,"log_size":48,"log_dups_size":0,"ondisk_log_size":48,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:27:32.110109+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0061513410000000003,"stat_sum":{"num_bytes":2050910,"num_objects":3,"num_object_clones":1,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":1,"num_read":288,"num_read_kb":3214,"num_write":54,"num_write_kb":1438,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"6","length":"5"}]},{"pgid":"3.b","version":"33'87","reported_seq":809,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-16T11:08:52.352279+0000","last_change":"2024-09-16T11:08:51.886242+0000","last_active":"2024-09-16T11:08:52.352279+0000","last_peered":"2024-09-16T11:08:52.352279+0000","last_clean":"2024-09-16T11:08:52.352279+0000","last_became_active":"2024-09-16T11:08:07.855380+0000","last_became_peered":"2024-09-16T11:08:07.855380+0000","last_unstale":"2024-09-16T11:08:52.352279+0000","last_undegraded":"2024-09-16T11:08:52.352279+0000","last_fullsized":"2024-09-16T11:08:52.352279+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:08:06.827173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:08:06.827173+0000","last_clean_scrub_stamp":"2024-09-16T11:08:06.827173+0000","objects_scrubbed":0,"log_size":87,"log_dups_size":0,"ondisk_log_size":87,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:48:03.017466+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0081692740000000007,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1188,"num_read_kb":4344,"num_write":97,"num_write_kb":1672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"6","length":"5"}]},{"pgid":"3.c","version":"32'70","reported_seq":398,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-16T11:08:51.881541+0000","last_change":"2024-09-16T11:08:51.881946+0000","last_active":"2024-09-16T11:08:51.881541+0000","last_peered":"2024-09-16T11:08:51.881541+0000","last_clean":"2024-09-16T11:08:51.881541+0000","last_became_active":"2024-09-16T11:08:07.859197+0000","last_became_peered":"2024-09-16T11:08:07.859197+0000","last_unstale":"2024-09-16T11:08:51.881541+0000","last_undegraded":"2024-09-16T11:08:51.881541+0000","last_fullsized":"2024-09-16T11:08:51.881541+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:08:06.827173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:08:06.827173+0000","last_clean_scrub_stamp":"2024-09-16T11:08:06.827173+0000","objects_scrubbed":0,"log_size":70,"log_dups_size":0,"ondisk_log_size":70,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:11:35.974778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019992830000000001,"stat_sum":{"num_bytes":3853000,"num_objects":3,"num_object_clones":1,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":1,"num_read":617,"num_read_kb":1509,"num_write":85,"num_write_kb":959,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"6","length":"5"}]},{"pgid":"3.d","version":"33'75","reported_seq":455,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-16T11:08:51.886372+0000","last_change":"2024-09-16T11:08:51.887845+0000","last_active":"2024-09-16T11:08:51.886372+0000","last_peered":"2024-09-16T11:08:51.886372+0000","last_clean":"2024-09-16T11:08:51.886372+0000","last_became_active":"2024-09-16T11:08:07.853865+0000","last_became_peered":"2024-09-16T11:08:07.853865+0000","last_unstale":"2024-09-16T11:08:51.886372+0000","last_undegraded":"2024-09-16T11:08:51.886372+0000","last_fullsized":"2024-09-16T11:08:51.886372+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:08:06.827173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:08:06.827173+0000","last_clean_scrub_stamp":"2024-09-16T11:08:06.827173+0000","objects_scrubbed":0,"log_size":75,"log_dups_size":0,"ondisk_log_size":75,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:35:17.633354+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0089916990000000006,"stat_sum":{"num_bytes":131072,"num_objects":2,"num_object_clones":1,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":785,"num_read_kb":1332,"num_write":69,"num_write_kb":661,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"6","length":"5"}]},{"pgid":"3.e","version":"33'52","reported_seq":312,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-16T11:08:51.881952+0000","last_change":"2024-09-16T11:08:51.881952+0000","last_active":"2024-09-16T11:08:51.881952+0000","last_peered":"2024-09-16T11:08:51.881952+0000","last_clean":"2024-09-16T11:08:51.881952+0000","last_became_active":"2024-09-16T11:08:07.847486+0000","last_became_peered":"2024-09-16T11:08:07.847486+0000","last_unstale":"2024-09-16T11:08:51.881952+0000","last_undegraded":"2024-09-16T11:08:51.881952+0000","last_fullsized":"2024-09-16T11:08:51.881952+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:08:06.827173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:08:06.827173+0000","last_clean_scrub_stamp":"2024-09-16T11:08:06.827173+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:40:57.767491+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.0067812660000000002,"stat_sum":{"num_bytes":196608,"num_objects":2,"num_object_clones":1,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":234,"num_read_kb":2061,"num_write":48,"num_write_kb":758,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"6","length":"5"}]},{"pgid":"3.f","version":"29'29","reported_seq":220,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-16T11:08:51.888411+0000","last_change":"2024-09-16T11:08:51.888577+0000","last_active":"2024-09-16T11:08:51.888411+0000","last_peered":"2024-09-16T11:08:51.888411+0000","last_clean":"2024-09-16T11:08:51.888411+0000","last_became_active":"2024-09-16T11:08:07.859480+0000","last_became_peered":"2024-09-16T11:08:07.859480+0000","last_unstale":"2024-09-16T11:08:51.888411+0000","last_undegraded":"2024-09-16T11:08:51.888411+0000","last_fullsized":"2024-09-16T11:08:51.888411+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:08:06.827173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:08:06.827173+0000","last_clean_scrub_stamp":"2024-09-16T11:08:06.827173+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:47:04.478053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0077316599999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":155,"num_read_kb":866,"num_write":30,"num_write_kb":312,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"6","length":"5"}]},{"pgid":"3.10","version":"33'33","reported_seq":160,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-16T11:08:51.882406+0000","last_change":"2024-09-16T11:08:51.882679+0000","last_active":"2024-09-16T11:08:51.882406+0000","last_peered":"2024-09-16T11:08:51.882406+0000","last_clean":"2024-09-16T11:08:51.882406+0000","last_became_active":"2024-09-16T11:08:07.855591+0000","last_became_peered":"2024-09-16T11:08:07.855591+0000","last_unstale":"2024-09-16T11:08:51.882406+0000","last_undegraded":"2024-09-16T11:08:51.882406+0000","last_fullsized":"2024-09-16T11:08:51.882406+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:08:06.827173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:08:06.827173+0000","last_clean_scrub_stamp":"2024-09-16T11:08:06.827173+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":0,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:20:16.980280+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0057037219999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":83,"num_read_kb":512,"num_write":38,"num_write_kb":467,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"6","length":"5"}]},{"pgid":"3.11","version":"33'88","reported_seq":947,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-16T11:08:52.203437+0000","last_change":"2024-09-16T11:08:51.878317+0000","last_active":"2024-09-16T11:08:52.203437+0000","last_peered":"2024-09-16T11:08:52.203437+0000","last_clean":"2024-09-16T11:08:52.203437+0000","last_became_active":"2024-09-16T11:08:07.856626+0000","last_became_peered":"2024-09-16T11:08:07.856626+0000","last_unstale":"2024-09-16T11:08:52.203437+0000","last_undegraded":"2024-09-16T11:08:52.203437+0000","last_fullsized":"2024-09-16T11:08:52.203437+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:08:06.827173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:08:06.827173+0000","last_clean_scrub_stamp":"2024-09-16T11:08:06.827173+0000","objects_scrubbed":0,"log_size":88,"log_dups_size":0,"ondisk_log_size":88,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:27:32.110109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0031249049999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1527,"num_read_kb":3994,"num_write":85,"num_write_kb":292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"6","length":"5"}]},{"pgid":"3.12","version":"33'184","reported_seq":362,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-16T11:08:52.354747+0000","last_change":"2024-09-16T11:08:51.881467+0000","last_active":"2024-09-16T11:08:52.354747+0000","last_peered":"2024-09-16T11:08:52.354747+0000","last_clean":"2024-09-16T11:08:52.354747+0000","last_became_active":"2024-09-16T11:08:07.857698+0000","last_became_peered":"2024-09-16T11:08:07.857698+0000","last_unstale":"2024-09-16T11:08:52.354747+0000","last_undegraded":"2024-09-16T11:08:52.354747+0000","last_fullsized":"2024-09-16T11:08:52.354747+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:08:06.827173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:08:06.827173+0000","last_clean_scrub_stamp":"2024-09-16T11:08:06.827173+0000","objects_scrubbed":0,"log_size":184,"log_dups_size":0,"ondisk_log_size":184,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:30:04.184361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016087269999999999,"stat_sum":{"num_bytes":327680,"num_objects":4,"num_object_clones":2,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":912,"num_read_kb":991,"num_write":286,"num_write_kb":407,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"6","length":"5"}]},{"pgid":"3.13","version":"32'228","reported_seq":679,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-16T11:08:51.881474+0000","last_change":"2024-09-16T11:08:51.881853+0000","last_active":"2024-09-16T11:08:51.881474+0000","last_peered":"2024-09-16T11:08:51.881474+0000","last_clean":"2024-09-16T11:08:51.881474+0000","last_became_active":"2024-09-16T11:08:07.853749+0000","last_became_peered":"2024-09-16T11:08:07.853749+0000","last_unstale":"2024-09-16T11:08:51.881474+0000","last_undegraded":"2024-09-16T11:08:51.881474+0000","last_fullsized":"2024-09-16T11:08:51.881474+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:08:06.827173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:08:06.827173+0000","last_clean_scrub_stamp":"2024-09-16T11:08:06.827173+0000","objects_scrubbed":0,"log_size":228,"log_dups_size":0,"ondisk_log_size":228,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:24:10.327154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0045125729999999998,"stat_sum":{"num_bytes":2710677,"num_objects":4,"num_object_clones":2,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":2,"num_read":1659,"num_read_kb":2488,"num_write":313,"num_write_kb":1774,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"6","length":"5"}]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":37543495,"num_objects":59,"num_object_clones":19,"num_object_copies":118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59,"num_whiteouts":7,"num_read":15654,"num_read_kb":48731,"num_write":2949,"num_write_kb":27896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1679360,"data_stored":1553218,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2346,"ondisk_log_size":2346,"up":64,"acting":64,"num_store_stats":3},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":56,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":52,"ondisk_log_size":52,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359738387,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302260,"kb_used_data":748,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94069580,"statfs":{"total":96636764160,"available":96327249920,"internally_reserved":0,"allocated":765952,"data_stored":576358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 11:08:40 2024","interfaces":[{"interface":"back","average":{"1min":0.221,"5min":0.221,"15min":0.221},"min":{"1min":0.14599999999999999,"5min":0.14599999999999999,"15min":0.14599999999999999},"max":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"last":0.28299999999999997},{"interface":"front","average":{"1min":0.23200000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"min":{"1min":0.17000000000000001,"5min":0.17000000000000001,"15min":0.17000000000000001},"max":{"1min":0.31,"5min":0.31,"15min":0.31},"last":0.24199999999999999}]},{"osd":1,"last update":"Mon Sep 16 11:08:40 2024","interfaces":[{"interface":"back","average":{"1min":0.23799999999999999,"5min":0.23799999999999999,"15min":0.23799999999999999},"min":{"1min":0.159,"5min":0.159,"15min":0.159},"max":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"last":0.22800000000000001},{"interface":"front","average":{"1min":0.252,"5min":0.252,"15min":0.252},"min":{"1min":0.184,"5min":0.184,"15min":0.184},"max":{"1min":0.376,"5min":0.376,"15min":0.376},"last":0.253}]}]},{"osd":1,"up_from":8,"seq":34359738388,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":308228,"kb_used_data":2548,"kb_used_omap":1,"kb_used_meta":305662,"kb_avail":94063612,"statfs":{"total":96636764160,"available":96321138688,"internally_reserved":0,"allocated":2609152,"data_stored":3203184,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":312998232},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 11:08:37 2024","interfaces":[{"interface":"back","average":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"min":{"1min":0.17399999999999999,"5min":0.17399999999999999,"15min":0.17399999999999999},"max":{"1min":0.376,"5min":0.376,"15min":0.376},"last":0.193},{"interface":"front","average":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"min":{"1min":0.17599999999999999,"5min":0.17599999999999999,"15min":0.17599999999999999},"max":{"1min":0.376,"5min":0.376,"15min":0.376},"last":0.20300000000000001}]},{"osd":2,"last update":"Mon Sep 16 11:08:37 2024","interfaces":[{"interface":"back","average":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"min":{"1min":0.16600000000000001,"5min":0.16600000000000001,"15min":0.16600000000000001},"max":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"last":0.183},{"interface":"front","average":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"min":{"1min":0.191,"5min":0.191,"15min":0.191},"max":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"last":0.217}]}]},{"osd":0,"up_from":8,"seq":34359738387,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":306944,"kb_used_data":1264,"kb_used_omap":1,"kb_used_meta":305662,"kb_avail":94064896,"statfs":{"total":96636764160,"available":96322453504,"internally_reserved":0,"allocated":1294336,"data_stored":1071344,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":312998232},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Sep 16 11:08:40 2024","interfaces":[{"interface":"back","average":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"min":{"1min":0.14599999999999999,"5min":0.14599999999999999,"15min":0.14599999999999999},"max":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"last":0.17899999999999999},{"interface":"front","average":{"1min":0.31,"5min":0.31,"15min":0.31},"min":{"1min":0.16600000000000001,"5min":0.16600000000000001,"15min":0.16600000000000001},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.20899999999999999}]},{"osd":2,"last update":"Mon Sep 16 11:08:40 2024","interfaces":[{"interface":"back","average":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"min":{"1min":0.20300000000000001,"5min":0.20300000000000001,"15min":0.20300000000000001},"max":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"last":0.19500000000000001},{"interface":"front","average":{"1min":0.309,"5min":0.309,"15min":0.309},"min":{"1min":0.186,"5min":0.186,"15min":0.186},"max":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"last":0.247}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":421888,"data_stored":374997,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":765952,"data_stored":707823,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":491520,"data_stored":470398,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T11:08:55.089 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T11:08:55.349 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-16T11:08:55.349 INFO:teuthology.orchestra.run.smithi102.stderr:dumped all 2024-09-16T11:08:55.360 INFO:teuthology.orchestra.run.smithi102.stdout:{"pg_ready":true,"pg_map":{"version":71,"stamp":"2024-09-16T11:08:53.880660+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":38133882,"num_objects":63,"num_object_clones":19,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63,"num_whiteouts":7,"num_read":15710,"num_read_kb":48777,"num_write":3030,"num_write_kb":29004,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2401,"ondisk_log_size":2401,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":917432,"kb_used_data":4560,"kb_used_omap":4,"kb_used_meta":912827,"kb_avail":282198088,"statfs":{"total":289910292480,"available":288970842112,"internally_reserved":0,"allocated":4669440,"data_stored":4850886,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":5112,"internal_metadata":934734856},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-161246497,"num_objects":-114,"num_object_clones":-24,"num_object_copies":-228,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-114,"num_whiteouts":-19,"num_read":1251,"num_read_kb":1071,"num_write":242,"num_write_kb":63,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.021852"},"pg_stats":[{"pgid":"3.1f","version":"33'33","reported_seq":199,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-16T11:08:51.888152+0000","last_change":"2024-09-16T11:08:51.888351+0000","last_active":"2024-09-16T11:08:51.888152+0000","last_peered":"2024-09-16T11:08:51.888152+0000","last_clean":"2024-09-16T11:08:51.888152+0000","last_became_active":"2024-09-16T11:08:07.859102+0000","last_became_peered":"2024-09-16T11:08:07.859102+0000","last_unstale":"2024-09-16T11:08:51.888152+0000","last_undegraded":"2024-09-16T11:08:51.888152+0000","last_fullsized":"2024-09-16T11:08:51.888152+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:08:06.827173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:08:06.827173+0000","last_clean_scrub_stamp":"2024-09-16T11:08:06.827173+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":0,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:21:53.809689+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.008213639,"stat_sum":{"num_bytes":2813157,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":127,"num_read_kb":1006,"num_write":34,"num_write_kb":861,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"6","length":"5"}]},{"pgid":"3.1e","version":"33'155","reported_seq":741,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-16T11:08:52.349329+0000","last_change":"2024-09-16T11:08:51.877347+0000","last_active":"2024-09-16T11:08:52.349329+0000","last_peered":"2024-09-16T11:08:52.349329+0000","last_clean":"2024-09-16T11:08:52.349329+0000","last_became_active":"2024-09-16T11:08:07.856092+0000","last_became_peered":"2024-09-16T11:08:07.856092+0000","last_unstale":"2024-09-16T11:08:52.349329+0000","last_undegraded":"2024-09-16T11:08:52.349329+0000","last_fullsized":"2024-09-16T11:08:52.349329+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:08:06.827173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:08:06.827173+0000","last_clean_scrub_stamp":"2024-09-16T11:08:06.827173+0000","objects_scrubbed":0,"log_size":155,"log_dups_size":0,"ondisk_log_size":155,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:02:44.147643+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0023786520000000002,"stat_sum":{"num_bytes":37,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1807,"num_read_kb":2532,"num_write":196,"num_write_kb":424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"6","length":"5"}]},{"pgid":"3.1d","version":"28'275","reported_seq":332,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-16T11:08:51.877757+0000","last_change":"2024-09-16T11:08:51.878043+0000","last_active":"2024-09-16T11:08:51.877757+0000","last_peered":"2024-09-16T11:08:51.877757+0000","last_clean":"2024-09-16T11:08:51.877757+0000","last_became_active":"2024-09-16T11:08:07.847513+0000","last_became_peered":"2024-09-16T11:08:07.847513+0000","last_unstale":"2024-09-16T11:08:51.877757+0000","last_undegraded":"2024-09-16T11:08:51.877757+0000","last_fullsized":"2024-09-16T11:08:51.877757+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:08:06.827173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:08:06.827173+0000","last_clean_scrub_stamp":"2024-09-16T11:08:06.827173+0000","objects_scrubbed":0,"log_size":275,"log_dups_size":0,"ondisk_log_size":275,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:51:39.065833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.003055496,"stat_sum":{"num_bytes":42,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1234,"num_read_kb":1012,"num_write":445,"num_write_kb":597,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"6","length":"5"}]},{"pgid":"3.1c","version":"33'33","reported_seq":213,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-16T11:08:52.109073+0000","last_change":"2024-09-16T11:08:51.882421+0000","last_active":"2024-09-16T11:08:52.109073+0000","last_peered":"2024-09-16T11:08:52.109073+0000","last_clean":"2024-09-16T11:08:52.109073+0000","last_became_active":"2024-09-16T11:08:07.855430+0000","last_became_peered":"2024-09-16T11:08:07.855430+0000","last_unstale":"2024-09-16T11:08:52.109073+0000","last_undegraded":"2024-09-16T11:08:52.109073+0000","last_fullsized":"2024-09-16T11:08:52.109073+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:08:06.827173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:08:06.827173+0000","last_clean_scrub_stamp":"2024-09-16T11:08:06.827173+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":0,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:28:07.715262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0048504100000000003,"stat_sum":{"num_bytes":1869110,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":167,"num_read_kb":250,"num_write":42,"num_write_kb":566,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"6","length":"5"}]},{"pgid":"3.1b","version":"33'160","reported_seq":840,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-16T11:08:51.888964+0000","last_change":"2024-09-16T11:08:51.889166+0000","last_active":"2024-09-16T11:08:51.888964+0000","last_peered":"2024-09-16T11:08:51.888964+0000","last_clean":"2024-09-16T11:08:51.888964+0000","last_became_active":"2024-09-16T11:08:07.856827+0000","last_became_peered":"2024-09-16T11:08:07.856827+0000","last_unstale":"2024-09-16T11:08:51.888964+0000","last_undegraded":"2024-09-16T11:08:51.888964+0000","last_fullsized":"2024-09-16T11:08:51.888964+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:08:06.827173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:08:06.827173+0000","last_clean_scrub_stamp":"2024-09-16T11:08:06.827173+0000","objects_scrubbed":0,"log_size":160,"log_dups_size":0,"ondisk_log_size":160,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:47:27.295935+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0088276499999999994,"stat_sum":{"num_bytes":590386,"num_objects":4,"num_object_clones":2,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":1,"num_read":1407,"num_read_kb":2244,"num_write":139,"num_write_kb":887,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"6","length":"5"}]},{"pgid":"3.1a","version":"28'13","reported_seq":374,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-16T11:08:52.316726+0000","last_change":"2024-09-16T11:08:51.887798+0000","last_active":"2024-09-16T11:08:52.316726+0000","last_peered":"2024-09-16T11:08:52.316726+0000","last_clean":"2024-09-16T11:08:52.316726+0000","last_became_active":"2024-09-16T11:08:07.853372+0000","last_became_peered":"2024-09-16T11:08:07.853372+0000","last_unstale":"2024-09-16T11:08:52.316726+0000","last_undegraded":"2024-09-16T11:08:52.316726+0000","last_fullsized":"2024-09-16T11:08:52.316726+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:08:06.827173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:08:06.827173+0000","last_clean_scrub_stamp":"2024-09-16T11:08:06.827173+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:21:53.809689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0089025300000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":315,"num_read_kb":333,"num_write":13,"num_write_kb":81,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"6","length":"5"}]},{"pgid":"3.19","version":"32'39","reported_seq":280,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-16T11:08:51.881126+0000","last_change":"2024-09-16T11:08:51.881384+0000","last_active":"2024-09-16T11:08:51.881126+0000","last_peered":"2024-09-16T11:08:51.881126+0000","last_clean":"2024-09-16T11:08:51.881126+0000","last_became_active":"2024-09-16T11:08:07.852874+0000","last_became_peered":"2024-09-16T11:08:07.852874+0000","last_unstale":"2024-09-16T11:08:51.881126+0000","last_undegraded":"2024-09-16T11:08:51.881126+0000","last_fullsized":"2024-09-16T11:08:51.881126+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:08:06.827173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:08:06.827173+0000","last_clean_scrub_stamp":"2024-09-16T11:08:06.827173+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:47:27.295935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0042733110000000001,"stat_sum":{"num_bytes":6202271,"num_objects":6,"num_object_clones":2,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":214,"num_read_kb":667,"num_write":42,"num_write_kb":746,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"6","length":"5"}]},{"pgid":"3.18","version":"28'17","reported_seq":66,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-16T11:08:51.874889+0000","last_change":"2024-09-16T11:08:51.875249+0000","last_active":"2024-09-16T11:08:51.874889+0000","last_peered":"2024-09-16T11:08:51.874889+0000","last_clean":"2024-09-16T11:08:51.874889+0000","last_became_active":"2024-09-16T11:08:07.848434+0000","last_became_peered":"2024-09-16T11:08:07.848434+0000","last_unstale":"2024-09-16T11:08:51.874889+0000","last_undegraded":"2024-09-16T11:08:51.874889+0000","last_fullsized":"2024-09-16T11:08:51.874889+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:08:06.827173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:08:06.827173+0000","last_clean_scrub_stamp":"2024-09-16T11:08:06.827173+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:47:04.478053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055608000000000003,"stat_sum":{"num_bytes":2342210,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":17,"num_write_kb":60,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"6","length":"5"}]},{"pgid":"3.17","version":"31'25","reported_seq":99,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-16T11:08:51.884747+0000","last_change":"2024-09-16T11:08:51.885023+0000","last_active":"2024-09-16T11:08:51.884747+0000","last_peered":"2024-09-16T11:08:51.884747+0000","last_clean":"2024-09-16T11:08:51.884747+0000","last_became_active":"2024-09-16T11:08:07.859097+0000","last_became_peered":"2024-09-16T11:08:07.859097+0000","last_unstale":"2024-09-16T11:08:51.884747+0000","last_undegraded":"2024-09-16T11:08:51.884747+0000","last_fullsized":"2024-09-16T11:08:51.884747+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:08:06.827173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:08:06.827173+0000","last_clean_scrub_stamp":"2024-09-16T11:08:06.827173+0000","objects_scrubbed":0,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:33:25.601273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0048877390000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":30,"num_read_kb":66,"num_write":24,"num_write_kb":325,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"6","length":"5"}]},{"pgid":"3.16","version":"32'25","reported_seq":128,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-16T11:08:51.878256+0000","last_change":"2024-09-16T11:08:51.878537+0000","last_active":"2024-09-16T11:08:51.878256+0000","last_peered":"2024-09-16T11:08:51.878256+0000","last_clean":"2024-09-16T11:08:51.878256+0000","last_became_active":"2024-09-16T11:08:07.856007+0000","last_became_peered":"2024-09-16T11:08:07.856007+0000","last_unstale":"2024-09-16T11:08:51.878256+0000","last_undegraded":"2024-09-16T11:08:51.878256+0000","last_fullsized":"2024-09-16T11:08:51.878256+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:08:06.827173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:08:06.827173+0000","last_clean_scrub_stamp":"2024-09-16T11:08:06.827173+0000","objects_scrubbed":0,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:18:30.121459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.003443495,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":66,"num_read_kb":1713,"num_write":38,"num_write_kb":1539,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"6","length":"5"}]},{"pgid":"3.15","version":"33'31","reported_seq":160,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-16T11:08:51.884698+0000","last_change":"2024-09-16T11:08:51.884698+0000","last_active":"2024-09-16T11:08:51.884698+0000","last_peered":"2024-09-16T11:08:51.884698+0000","last_clean":"2024-09-16T11:08:51.884698+0000","last_became_active":"2024-09-16T11:08:07.858375+0000","last_became_peered":"2024-09-16T11:08:07.858375+0000","last_unstale":"2024-09-16T11:08:51.884698+0000","last_undegraded":"2024-09-16T11:08:51.884698+0000","last_fullsized":"2024-09-16T11:08:51.884698+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:08:06.827173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:08:06.827173+0000","last_clean_scrub_stamp":"2024-09-16T11:08:06.827173+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:46:06.267425+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.0047413910000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":92,"num_read_kb":770,"num_write":36,"num_write_kb":1093,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"6","length":"5"}]},{"pgid":"3.14","version":"32'137","reported_seq":289,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-16T11:08:51.882715+0000","last_change":"2024-09-16T11:08:51.883437+0000","last_active":"2024-09-16T11:08:51.882715+0000","last_peered":"2024-09-16T11:08:51.882715+0000","last_clean":"2024-09-16T11:08:51.882715+0000","last_became_active":"2024-09-16T11:08:07.853189+0000","last_became_peered":"2024-09-16T11:08:07.853189+0000","last_unstale":"2024-09-16T11:08:51.882715+0000","last_undegraded":"2024-09-16T11:08:51.882715+0000","last_fullsized":"2024-09-16T11:08:51.882715+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:08:06.827173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:08:06.827173+0000","last_clean_scrub_stamp":"2024-09-16T11:08:06.827173+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:30:04.184361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0057208049999999998,"stat_sum":{"num_bytes":1600291,"num_objects":3,"num_object_clones":1,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":523,"num_read_kb":930,"num_write":187,"num_write_kb":943,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"6","length":"5"}]},{"pgid":"2.7","version":"0'0","reported_seq":33,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-16T11:08:50.865069+0000","last_change":"2024-09-16T11:07:46.454524+0000","last_active":"2024-09-16T11:08:50.865069+0000","last_peered":"2024-09-16T11:08:50.865069+0000","last_clean":"2024-09-16T11:08:50.865069+0000","last_became_active":"2024-09-16T11:07:44.446646+0000","last_became_peered":"2024-09-16T11:07:44.446646+0000","last_unstale":"2024-09-16T11:08:50.865069+0000","last_undegraded":"2024-09-16T11:08:50.865069+0000","last_fullsized":"2024-09-16T11:08:50.865069+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:07:43.427920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:07:43.427920+0000","last_clean_scrub_stamp":"2024-09-16T11:07:43.427920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:34:38.920978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000337344,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.6","version":"33'80","reported_seq":135,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-16T11:08:51.886576+0000","last_change":"2024-09-16T11:08:51.886576+0000","last_active":"2024-09-16T11:08:51.886576+0000","last_peered":"2024-09-16T11:08:51.886576+0000","last_clean":"2024-09-16T11:08:51.886576+0000","last_became_active":"2024-09-16T11:08:07.858208+0000","last_became_peered":"2024-09-16T11:08:07.858208+0000","last_unstale":"2024-09-16T11:08:51.886576+0000","last_undegraded":"2024-09-16T11:08:51.886576+0000","last_fullsized":"2024-09-16T11:08:51.886576+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:08:06.827173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:08:06.827173+0000","last_clean_scrub_stamp":"2024-09-16T11:08:06.827173+0000","objects_scrubbed":0,"log_size":80,"log_dups_size":0,"ondisk_log_size":80,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:20:16.980280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0063318970000000004,"stat_sum":{"num_bytes":3631436,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":259,"num_read_kb":213,"num_write":113,"num_write_kb":675,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"6","length":"5"}]},{"pgid":"2.6","version":"0'0","reported_seq":33,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-16T11:08:50.864872+0000","last_change":"2024-09-16T11:07:46.454822+0000","last_active":"2024-09-16T11:08:50.864872+0000","last_peered":"2024-09-16T11:08:50.864872+0000","last_clean":"2024-09-16T11:08:50.864872+0000","last_became_active":"2024-09-16T11:07:44.446406+0000","last_became_peered":"2024-09-16T11:07:44.446406+0000","last_unstale":"2024-09-16T11:08:50.864872+0000","last_undegraded":"2024-09-16T11:08:50.864872+0000","last_fullsized":"2024-09-16T11:08:50.864872+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:07:43.427920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:07:43.427920+0000","last_clean_scrub_stamp":"2024-09-16T11:07:43.427920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:12:45.650225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062301399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"31'41","reported_seq":314,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-16T11:08:51.878097+0000","last_change":"2024-09-16T11:08:51.878390+0000","last_active":"2024-09-16T11:08:51.878097+0000","last_peered":"2024-09-16T11:08:51.878097+0000","last_clean":"2024-09-16T11:08:51.878097+0000","last_became_active":"2024-09-16T11:08:07.856393+0000","last_became_peered":"2024-09-16T11:08:07.856393+0000","last_unstale":"2024-09-16T11:08:51.878097+0000","last_undegraded":"2024-09-16T11:08:51.878097+0000","last_fullsized":"2024-09-16T11:08:51.878097+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:08:06.827173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:08:06.827173+0000","last_clean_scrub_stamp":"2024-09-16T11:08:06.827173+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:11:35.974778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.003312602,"stat_sum":{"num_bytes":4131275,"num_objects":3,"num_object_clones":1,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":1,"num_read":241,"num_read_kb":2845,"num_write":54,"num_write_kb":2044,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"6","length":"5"}]},{"pgid":"2.5","version":"0'0","reported_seq":33,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-16T11:08:50.864449+0000","last_change":"2024-09-16T11:07:46.455055+0000","last_active":"2024-09-16T11:08:50.864449+0000","last_peered":"2024-09-16T11:08:50.864449+0000","last_clean":"2024-09-16T11:08:50.864449+0000","last_became_active":"2024-09-16T11:07:44.446498+0000","last_became_peered":"2024-09-16T11:07:44.446498+0000","last_unstale":"2024-09-16T11:08:50.864449+0000","last_undegraded":"2024-09-16T11:08:50.864449+0000","last_fullsized":"2024-09-16T11:08:50.864449+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:07:43.427920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:07:43.427920+0000","last_clean_scrub_stamp":"2024-09-16T11:07:43.427920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:40:08.726582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00084251400000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"32'67","reported_seq":266,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-16T11:08:51.883487+0000","last_change":"2024-09-16T11:08:51.883852+0000","last_active":"2024-09-16T11:08:51.883487+0000","last_peered":"2024-09-16T11:08:51.883487+0000","last_clean":"2024-09-16T11:08:51.883487+0000","last_became_active":"2024-09-16T11:08:07.853470+0000","last_became_peered":"2024-09-16T11:08:07.853470+0000","last_unstale":"2024-09-16T11:08:51.883487+0000","last_undegraded":"2024-09-16T11:08:51.883487+0000","last_fullsized":"2024-09-16T11:08:51.883487+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:08:06.827173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:08:06.827173+0000","last_clean_scrub_stamp":"2024-09-16T11:08:06.827173+0000","objects_scrubbed":0,"log_size":67,"log_dups_size":0,"ondisk_log_size":67,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:46:06.267425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0059983930000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":360,"num_read_kb":1468,"num_write":100,"num_write_kb":888,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"6","length":"5"}]},{"pgid":"2.4","version":"0'0","reported_seq":33,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-16T11:08:50.864582+0000","last_change":"2024-09-16T11:07:46.454774+0000","last_active":"2024-09-16T11:08:50.864582+0000","last_peered":"2024-09-16T11:08:50.864582+0000","last_clean":"2024-09-16T11:08:50.864582+0000","last_became_active":"2024-09-16T11:07:44.446413+0000","last_became_peered":"2024-09-16T11:07:44.446413+0000","last_unstale":"2024-09-16T11:08:50.864582+0000","last_undegraded":"2024-09-16T11:08:50.864582+0000","last_fullsized":"2024-09-16T11:08:50.864582+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:07:43.427920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:07:43.427920+0000","last_clean_scrub_stamp":"2024-09-16T11:07:43.427920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:53:07.204424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059042800000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"32'69","reported_seq":469,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-16T11:08:51.878615+0000","last_change":"2024-09-16T11:08:51.879403+0000","last_active":"2024-09-16T11:08:51.878615+0000","last_peered":"2024-09-16T11:08:51.878615+0000","last_clean":"2024-09-16T11:08:51.878615+0000","last_became_active":"2024-09-16T11:08:07.855280+0000","last_became_peered":"2024-09-16T11:08:07.855280+0000","last_unstale":"2024-09-16T11:08:51.878615+0000","last_undegraded":"2024-09-16T11:08:51.878615+0000","last_fullsized":"2024-09-16T11:08:51.878615+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:08:06.827173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:08:06.827173+0000","last_clean_scrub_stamp":"2024-09-16T11:08:06.827173+0000","objects_scrubbed":0,"log_size":69,"log_dups_size":0,"ondisk_log_size":69,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:03:42.315998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0042484180000000003,"stat_sum":{"num_bytes":786432,"num_objects":6,"num_object_clones":3,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":396,"num_read_kb":2349,"num_write":76,"num_write_kb":1069,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"6","length":"5"}]},{"pgid":"3.3","version":"33'78","reported_seq":247,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-16T11:08:52.210563+0000","last_change":"2024-09-16T11:08:51.886868+0000","last_active":"2024-09-16T11:08:52.210563+0000","last_peered":"2024-09-16T11:08:52.210563+0000","last_clean":"2024-09-16T11:08:52.210563+0000","last_became_active":"2024-09-16T11:08:07.847550+0000","last_became_peered":"2024-09-16T11:08:07.847550+0000","last_unstale":"2024-09-16T11:08:52.210563+0000","last_undegraded":"2024-09-16T11:08:52.210563+0000","last_fullsized":"2024-09-16T11:08:52.210563+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:08:06.827173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:08:06.827173+0000","last_clean_scrub_stamp":"2024-09-16T11:08:06.827173+0000","objects_scrubbed":0,"log_size":78,"log_dups_size":0,"ondisk_log_size":78,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:25:26.863744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0065329400000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":152,"num_read_kb":956,"num_write":108,"num_write_kb":2573,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"6","length":"5"}]},{"pgid":"2.2","version":"15'2","reported_seq":34,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-16T11:08:50.869072+0000","last_change":"2024-09-16T11:07:46.448909+0000","last_active":"2024-09-16T11:08:50.869072+0000","last_peered":"2024-09-16T11:08:50.869072+0000","last_clean":"2024-09-16T11:08:50.869072+0000","last_became_active":"2024-09-16T11:07:44.449069+0000","last_became_peered":"2024-09-16T11:07:44.449069+0000","last_unstale":"2024-09-16T11:08:50.869072+0000","last_undegraded":"2024-09-16T11:08:50.869072+0000","last_fullsized":"2024-09-16T11:08:50.869072+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:07:43.427920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:07:43.427920+0000","last_clean_scrub_stamp":"2024-09-16T11:07:43.427920+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:13:44.990400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00029305100000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.0","version":"33'46","reported_seq":203,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-16T11:08:52.184769+0000","last_change":"2024-09-16T11:08:51.881057+0000","last_active":"2024-09-16T11:08:52.184769+0000","last_peered":"2024-09-16T11:08:52.184769+0000","last_clean":"2024-09-16T11:08:52.184769+0000","last_became_active":"2024-09-16T11:08:07.855258+0000","last_became_peered":"2024-09-16T11:08:07.855258+0000","last_unstale":"2024-09-16T11:08:52.184769+0000","last_undegraded":"2024-09-16T11:08:52.184769+0000","last_fullsized":"2024-09-16T11:08:52.184769+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:08:06.827173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:08:06.827173+0000","last_clean_scrub_stamp":"2024-09-16T11:08:06.827173+0000","objects_scrubbed":0,"log_size":46,"log_dups_size":0,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:26:07.701673+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.0040037609999999998,"stat_sum":{"num_bytes":42,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":139,"num_read_kb":1672,"num_write":52,"num_write_kb":1263,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"6","length":"5"}]},{"pgid":"2.1","version":"0'0","reported_seq":32,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-16T11:08:50.861810+0000","last_change":"2024-09-16T11:07:46.720289+0000","last_active":"2024-09-16T11:08:50.861810+0000","last_peered":"2024-09-16T11:08:50.861810+0000","last_clean":"2024-09-16T11:08:50.861810+0000","last_became_active":"2024-09-16T11:07:44.447533+0000","last_became_peered":"2024-09-16T11:07:44.447533+0000","last_unstale":"2024-09-16T11:08:50.861810+0000","last_undegraded":"2024-09-16T11:08:50.861810+0000","last_fullsized":"2024-09-16T11:08:50.861810+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:07:43.427920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:07:43.427920+0000","last_clean_scrub_stamp":"2024-09-16T11:07:43.427920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:13:44.990400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000230378,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"33'30","reported_seq":160,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-16T11:08:51.882164+0000","last_change":"2024-09-16T11:08:51.883251+0000","last_active":"2024-09-16T11:08:51.882164+0000","last_peered":"2024-09-16T11:08:51.882164+0000","last_clean":"2024-09-16T11:08:51.882164+0000","last_became_active":"2024-09-16T11:08:07.847955+0000","last_became_peered":"2024-09-16T11:08:07.847955+0000","last_unstale":"2024-09-16T11:08:51.882164+0000","last_undegraded":"2024-09-16T11:08:51.882164+0000","last_fullsized":"2024-09-16T11:08:51.882164+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:08:06.827173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:08:06.827173+0000","last_clean_scrub_stamp":"2024-09-16T11:08:06.827173+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:02:44.147643+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.003530334,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":87,"num_read_kb":1164,"num_write":31,"num_write_kb":740,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"6","length":"5"}]},{"pgid":"2.0","version":"0'0","reported_seq":32,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-16T11:08:50.861284+0000","last_change":"2024-09-16T11:07:46.720264+0000","last_active":"2024-09-16T11:08:50.861284+0000","last_peered":"2024-09-16T11:08:50.861284+0000","last_clean":"2024-09-16T11:08:50.861284+0000","last_became_active":"2024-09-16T11:07:44.448153+0000","last_became_peered":"2024-09-16T11:07:44.448153+0000","last_unstale":"2024-09-16T11:08:50.861284+0000","last_undegraded":"2024-09-16T11:08:50.861284+0000","last_fullsized":"2024-09-16T11:08:50.861284+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:07:43.427920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:07:43.427920+0000","last_clean_scrub_stamp":"2024-09-16T11:07:43.427920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:09:42.536129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00024792500000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"33'40","reported_seq":266,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-16T11:08:52.254010+0000","last_change":"2024-09-16T11:08:51.889257+0000","last_active":"2024-09-16T11:08:52.254010+0000","last_peered":"2024-09-16T11:08:52.254010+0000","last_clean":"2024-09-16T11:08:52.254010+0000","last_became_active":"2024-09-16T11:08:07.853307+0000","last_became_peered":"2024-09-16T11:08:07.853307+0000","last_unstale":"2024-09-16T11:08:52.254010+0000","last_undegraded":"2024-09-16T11:08:52.254010+0000","last_fullsized":"2024-09-16T11:08:52.254010+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:08:06.827173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:08:06.827173+0000","last_clean_scrub_stamp":"2024-09-16T11:08:06.827173+0000","objects_scrubbed":0,"log_size":40,"log_dups_size":0,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:33:25.601273+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.010809077,"stat_sum":{"num_bytes":1243751,"num_objects":5,"num_object_clones":2,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":1,"num_read":203,"num_read_kb":1989,"num_write":37,"num_write_kb":217,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"6","length":"5"}]},{"pgid":"2.3","version":"13'1","reported_seq":34,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-16T11:08:50.865131+0000","last_change":"2024-09-16T11:07:46.454516+0000","last_active":"2024-09-16T11:08:50.865131+0000","last_peered":"2024-09-16T11:08:50.865131+0000","last_clean":"2024-09-16T11:08:50.865131+0000","last_became_active":"2024-09-16T11:07:44.446367+0000","last_became_peered":"2024-09-16T11:07:44.446367+0000","last_unstale":"2024-09-16T11:08:50.865131+0000","last_undegraded":"2024-09-16T11:08:50.865131+0000","last_fullsized":"2024-09-16T11:08:50.865131+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:07:43.427920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:07:43.427920+0000","last_clean_scrub_stamp":"2024-09-16T11:07:43.427920+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:06:21.202391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00038771200000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"10'52","reported_seq":98,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-16T11:08:50.864840+0000","last_change":"2024-09-16T11:07:39.420884+0000","last_active":"2024-09-16T11:08:50.864840+0000","last_peered":"2024-09-16T11:08:50.864840+0000","last_clean":"2024-09-16T11:08:50.864840+0000","last_became_active":"2024-09-16T11:07:39.420564+0000","last_became_peered":"2024-09-16T11:07:39.420564+0000","last_unstale":"2024-09-16T11:08:50.864840+0000","last_undegraded":"2024-09-16T11:08:50.864840+0000","last_fullsized":"2024-09-16T11:08:50.864840+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:07:38.411971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:07:38.411971+0000","last_clean_scrub_stamp":"2024-09-16T11:07:38.411971+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:52:50.808244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":56,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"33'41","reported_seq":448,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-16T11:08:51.877660+0000","last_change":"2024-09-16T11:08:51.877660+0000","last_active":"2024-09-16T11:08:51.877660+0000","last_peered":"2024-09-16T11:08:51.877660+0000","last_clean":"2024-09-16T11:08:51.877660+0000","last_became_active":"2024-09-16T11:08:07.857280+0000","last_became_peered":"2024-09-16T11:08:07.857280+0000","last_unstale":"2024-09-16T11:08:51.877660+0000","last_undegraded":"2024-09-16T11:08:51.877660+0000","last_fullsized":"2024-09-16T11:08:51.877660+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:08:06.827173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:08:06.827173+0000","last_clean_scrub_stamp":"2024-09-16T11:08:06.827173+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:25:26.863744+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.0028720669999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":381,"num_read_kb":3183,"num_write":40,"num_write_kb":936,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"6","length":"5"}]},{"pgid":"3.9","version":"28'17","reported_seq":65,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-16T11:08:51.879719+0000","last_change":"2024-09-16T11:08:51.880725+0000","last_active":"2024-09-16T11:08:51.879719+0000","last_peered":"2024-09-16T11:08:51.879719+0000","last_clean":"2024-09-16T11:08:51.879719+0000","last_became_active":"2024-09-16T11:08:07.850403+0000","last_became_peered":"2024-09-16T11:08:07.850403+0000","last_unstale":"2024-09-16T11:08:51.879719+0000","last_undegraded":"2024-09-16T11:08:51.879719+0000","last_fullsized":"2024-09-16T11:08:51.879719+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:08:06.827173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:08:06.827173+0000","last_clean_scrub_stamp":"2024-09-16T11:08:06.827173+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:32:06.761762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011094919999999999,"stat_sum":{"num_bytes":3063089,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":3,"num_read_kb":55,"num_write":20,"num_write_kb":629,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"6","length":"5"}]},{"pgid":"3.a","version":"33'48","reported_seq":361,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-16T11:08:51.886253+0000","last_change":"2024-09-16T11:08:51.886253+0000","last_active":"2024-09-16T11:08:51.886253+0000","last_peered":"2024-09-16T11:08:51.886253+0000","last_clean":"2024-09-16T11:08:51.886253+0000","last_became_active":"2024-09-16T11:08:07.857407+0000","last_became_peered":"2024-09-16T11:08:07.857407+0000","last_unstale":"2024-09-16T11:08:51.886253+0000","last_undegraded":"2024-09-16T11:08:51.886253+0000","last_fullsized":"2024-09-16T11:08:51.886253+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:08:06.827173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:08:06.827173+0000","last_clean_scrub_stamp":"2024-09-16T11:08:06.827173+0000","objects_scrubbed":0,"log_size":48,"log_dups_size":0,"ondisk_log_size":48,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:27:32.110109+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0061513410000000003,"stat_sum":{"num_bytes":2050910,"num_objects":3,"num_object_clones":1,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":1,"num_read":288,"num_read_kb":3214,"num_write":54,"num_write_kb":1438,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"6","length":"5"}]},{"pgid":"3.b","version":"33'87","reported_seq":809,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-16T11:08:52.352279+0000","last_change":"2024-09-16T11:08:51.886242+0000","last_active":"2024-09-16T11:08:52.352279+0000","last_peered":"2024-09-16T11:08:52.352279+0000","last_clean":"2024-09-16T11:08:52.352279+0000","last_became_active":"2024-09-16T11:08:07.855380+0000","last_became_peered":"2024-09-16T11:08:07.855380+0000","last_unstale":"2024-09-16T11:08:52.352279+0000","last_undegraded":"2024-09-16T11:08:52.352279+0000","last_fullsized":"2024-09-16T11:08:52.352279+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:08:06.827173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:08:06.827173+0000","last_clean_scrub_stamp":"2024-09-16T11:08:06.827173+0000","objects_scrubbed":0,"log_size":87,"log_dups_size":0,"ondisk_log_size":87,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:48:03.017466+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0081692740000000007,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1188,"num_read_kb":4344,"num_write":97,"num_write_kb":1672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"6","length":"5"}]},{"pgid":"3.c","version":"32'70","reported_seq":398,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-16T11:08:51.881541+0000","last_change":"2024-09-16T11:08:51.881946+0000","last_active":"2024-09-16T11:08:51.881541+0000","last_peered":"2024-09-16T11:08:51.881541+0000","last_clean":"2024-09-16T11:08:51.881541+0000","last_became_active":"2024-09-16T11:08:07.859197+0000","last_became_peered":"2024-09-16T11:08:07.859197+0000","last_unstale":"2024-09-16T11:08:51.881541+0000","last_undegraded":"2024-09-16T11:08:51.881541+0000","last_fullsized":"2024-09-16T11:08:51.881541+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:08:06.827173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:08:06.827173+0000","last_clean_scrub_stamp":"2024-09-16T11:08:06.827173+0000","objects_scrubbed":0,"log_size":70,"log_dups_size":0,"ondisk_log_size":70,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:11:35.974778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019992830000000001,"stat_sum":{"num_bytes":3853000,"num_objects":3,"num_object_clones":1,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":1,"num_read":617,"num_read_kb":1509,"num_write":85,"num_write_kb":959,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"6","length":"5"}]},{"pgid":"3.d","version":"33'75","reported_seq":455,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-16T11:08:51.886372+0000","last_change":"2024-09-16T11:08:51.887845+0000","last_active":"2024-09-16T11:08:51.886372+0000","last_peered":"2024-09-16T11:08:51.886372+0000","last_clean":"2024-09-16T11:08:51.886372+0000","last_became_active":"2024-09-16T11:08:07.853865+0000","last_became_peered":"2024-09-16T11:08:07.853865+0000","last_unstale":"2024-09-16T11:08:51.886372+0000","last_undegraded":"2024-09-16T11:08:51.886372+0000","last_fullsized":"2024-09-16T11:08:51.886372+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:08:06.827173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:08:06.827173+0000","last_clean_scrub_stamp":"2024-09-16T11:08:06.827173+0000","objects_scrubbed":0,"log_size":75,"log_dups_size":0,"ondisk_log_size":75,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:35:17.633354+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0089916990000000006,"stat_sum":{"num_bytes":131072,"num_objects":2,"num_object_clones":1,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":785,"num_read_kb":1332,"num_write":69,"num_write_kb":661,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"6","length":"5"}]},{"pgid":"3.e","version":"33'52","reported_seq":312,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-16T11:08:51.881952+0000","last_change":"2024-09-16T11:08:51.881952+0000","last_active":"2024-09-16T11:08:51.881952+0000","last_peered":"2024-09-16T11:08:51.881952+0000","last_clean":"2024-09-16T11:08:51.881952+0000","last_became_active":"2024-09-16T11:08:07.847486+0000","last_became_peered":"2024-09-16T11:08:07.847486+0000","last_unstale":"2024-09-16T11:08:51.881952+0000","last_undegraded":"2024-09-16T11:08:51.881952+0000","last_fullsized":"2024-09-16T11:08:51.881952+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:08:06.827173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:08:06.827173+0000","last_clean_scrub_stamp":"2024-09-16T11:08:06.827173+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:40:57.767491+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.0067812660000000002,"stat_sum":{"num_bytes":196608,"num_objects":2,"num_object_clones":1,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":234,"num_read_kb":2061,"num_write":48,"num_write_kb":758,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"6","length":"5"}]},{"pgid":"3.f","version":"29'29","reported_seq":220,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-16T11:08:51.888411+0000","last_change":"2024-09-16T11:08:51.888577+0000","last_active":"2024-09-16T11:08:51.888411+0000","last_peered":"2024-09-16T11:08:51.888411+0000","last_clean":"2024-09-16T11:08:51.888411+0000","last_became_active":"2024-09-16T11:08:07.859480+0000","last_became_peered":"2024-09-16T11:08:07.859480+0000","last_unstale":"2024-09-16T11:08:51.888411+0000","last_undegraded":"2024-09-16T11:08:51.888411+0000","last_fullsized":"2024-09-16T11:08:51.888411+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:08:06.827173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:08:06.827173+0000","last_clean_scrub_stamp":"2024-09-16T11:08:06.827173+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:47:04.478053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0077316599999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":155,"num_read_kb":866,"num_write":30,"num_write_kb":312,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"6","length":"5"}]},{"pgid":"3.10","version":"33'33","reported_seq":160,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-16T11:08:51.882406+0000","last_change":"2024-09-16T11:08:51.882679+0000","last_active":"2024-09-16T11:08:51.882406+0000","last_peered":"2024-09-16T11:08:51.882406+0000","last_clean":"2024-09-16T11:08:51.882406+0000","last_became_active":"2024-09-16T11:08:07.855591+0000","last_became_peered":"2024-09-16T11:08:07.855591+0000","last_unstale":"2024-09-16T11:08:51.882406+0000","last_undegraded":"2024-09-16T11:08:51.882406+0000","last_fullsized":"2024-09-16T11:08:51.882406+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:08:06.827173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:08:06.827173+0000","last_clean_scrub_stamp":"2024-09-16T11:08:06.827173+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":0,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:20:16.980280+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0057037219999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":83,"num_read_kb":512,"num_write":38,"num_write_kb":467,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"6","length":"5"}]},{"pgid":"3.11","version":"33'88","reported_seq":947,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-16T11:08:52.203437+0000","last_change":"2024-09-16T11:08:51.878317+0000","last_active":"2024-09-16T11:08:52.203437+0000","last_peered":"2024-09-16T11:08:52.203437+0000","last_clean":"2024-09-16T11:08:52.203437+0000","last_became_active":"2024-09-16T11:08:07.856626+0000","last_became_peered":"2024-09-16T11:08:07.856626+0000","last_unstale":"2024-09-16T11:08:52.203437+0000","last_undegraded":"2024-09-16T11:08:52.203437+0000","last_fullsized":"2024-09-16T11:08:52.203437+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:08:06.827173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:08:06.827173+0000","last_clean_scrub_stamp":"2024-09-16T11:08:06.827173+0000","objects_scrubbed":0,"log_size":88,"log_dups_size":0,"ondisk_log_size":88,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:27:32.110109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0031249049999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1527,"num_read_kb":3994,"num_write":85,"num_write_kb":292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"6","length":"5"}]},{"pgid":"3.12","version":"33'184","reported_seq":362,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-16T11:08:52.354747+0000","last_change":"2024-09-16T11:08:51.881467+0000","last_active":"2024-09-16T11:08:52.354747+0000","last_peered":"2024-09-16T11:08:52.354747+0000","last_clean":"2024-09-16T11:08:52.354747+0000","last_became_active":"2024-09-16T11:08:07.857698+0000","last_became_peered":"2024-09-16T11:08:07.857698+0000","last_unstale":"2024-09-16T11:08:52.354747+0000","last_undegraded":"2024-09-16T11:08:52.354747+0000","last_fullsized":"2024-09-16T11:08:52.354747+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:08:06.827173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:08:06.827173+0000","last_clean_scrub_stamp":"2024-09-16T11:08:06.827173+0000","objects_scrubbed":0,"log_size":184,"log_dups_size":0,"ondisk_log_size":184,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:30:04.184361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016087269999999999,"stat_sum":{"num_bytes":327680,"num_objects":4,"num_object_clones":2,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":912,"num_read_kb":991,"num_write":286,"num_write_kb":407,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"6","length":"5"}]},{"pgid":"3.13","version":"32'228","reported_seq":679,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-16T11:08:51.881474+0000","last_change":"2024-09-16T11:08:51.881853+0000","last_active":"2024-09-16T11:08:51.881474+0000","last_peered":"2024-09-16T11:08:51.881474+0000","last_clean":"2024-09-16T11:08:51.881474+0000","last_became_active":"2024-09-16T11:08:07.853749+0000","last_became_peered":"2024-09-16T11:08:07.853749+0000","last_unstale":"2024-09-16T11:08:51.881474+0000","last_undegraded":"2024-09-16T11:08:51.881474+0000","last_fullsized":"2024-09-16T11:08:51.881474+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:08:06.827173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:08:06.827173+0000","last_clean_scrub_stamp":"2024-09-16T11:08:06.827173+0000","objects_scrubbed":0,"log_size":228,"log_dups_size":0,"ondisk_log_size":228,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:24:10.327154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0045125729999999998,"stat_sum":{"num_bytes":2710677,"num_objects":4,"num_object_clones":2,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":2,"num_read":1659,"num_read_kb":2488,"num_write":313,"num_write_kb":1774,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"6","length":"5"}]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":37543495,"num_objects":59,"num_object_clones":19,"num_object_copies":118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59,"num_whiteouts":7,"num_read":15654,"num_read_kb":48731,"num_write":2949,"num_write_kb":27896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1679360,"data_stored":1553218,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2346,"ondisk_log_size":2346,"up":64,"acting":64,"num_store_stats":3},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":56,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":52,"ondisk_log_size":52,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359738387,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302260,"kb_used_data":748,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94069580,"statfs":{"total":96636764160,"available":96327249920,"internally_reserved":0,"allocated":765952,"data_stored":576358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 11:08:40 2024","interfaces":[{"interface":"back","average":{"1min":0.221,"5min":0.221,"15min":0.221},"min":{"1min":0.14599999999999999,"5min":0.14599999999999999,"15min":0.14599999999999999},"max":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"last":0.28299999999999997},{"interface":"front","average":{"1min":0.23200000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"min":{"1min":0.17000000000000001,"5min":0.17000000000000001,"15min":0.17000000000000001},"max":{"1min":0.31,"5min":0.31,"15min":0.31},"last":0.24199999999999999}]},{"osd":1,"last update":"Mon Sep 16 11:08:40 2024","interfaces":[{"interface":"back","average":{"1min":0.23799999999999999,"5min":0.23799999999999999,"15min":0.23799999999999999},"min":{"1min":0.159,"5min":0.159,"15min":0.159},"max":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"last":0.22800000000000001},{"interface":"front","average":{"1min":0.252,"5min":0.252,"15min":0.252},"min":{"1min":0.184,"5min":0.184,"15min":0.184},"max":{"1min":0.376,"5min":0.376,"15min":0.376},"last":0.253}]}]},{"osd":1,"up_from":8,"seq":34359738388,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":308228,"kb_used_data":2548,"kb_used_omap":1,"kb_used_meta":305662,"kb_avail":94063612,"statfs":{"total":96636764160,"available":96321138688,"internally_reserved":0,"allocated":2609152,"data_stored":3203184,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":312998232},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 11:08:37 2024","interfaces":[{"interface":"back","average":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"min":{"1min":0.17399999999999999,"5min":0.17399999999999999,"15min":0.17399999999999999},"max":{"1min":0.376,"5min":0.376,"15min":0.376},"last":0.193},{"interface":"front","average":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"min":{"1min":0.17599999999999999,"5min":0.17599999999999999,"15min":0.17599999999999999},"max":{"1min":0.376,"5min":0.376,"15min":0.376},"last":0.20300000000000001}]},{"osd":2,"last update":"Mon Sep 16 11:08:37 2024","interfaces":[{"interface":"back","average":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"min":{"1min":0.16600000000000001,"5min":0.16600000000000001,"15min":0.16600000000000001},"max":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"last":0.183},{"interface":"front","average":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"min":{"1min":0.191,"5min":0.191,"15min":0.191},"max":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"last":0.217}]}]},{"osd":0,"up_from":8,"seq":34359738387,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":306944,"kb_used_data":1264,"kb_used_omap":1,"kb_used_meta":305662,"kb_avail":94064896,"statfs":{"total":96636764160,"available":96322453504,"internally_reserved":0,"allocated":1294336,"data_stored":1071344,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":312998232},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Sep 16 11:08:40 2024","interfaces":[{"interface":"back","average":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"min":{"1min":0.14599999999999999,"5min":0.14599999999999999,"15min":0.14599999999999999},"max":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"last":0.17899999999999999},{"interface":"front","average":{"1min":0.31,"5min":0.31,"15min":0.31},"min":{"1min":0.16600000000000001,"5min":0.16600000000000001,"15min":0.16600000000000001},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.20899999999999999}]},{"osd":2,"last update":"Mon Sep 16 11:08:40 2024","interfaces":[{"interface":"back","average":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"min":{"1min":0.20300000000000001,"5min":0.20300000000000001,"15min":0.20300000000000001},"max":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"last":0.19500000000000001},{"interface":"front","average":{"1min":0.309,"5min":0.309,"15min":0.309},"min":{"1min":0.186,"5min":0.186,"15min":0.186},"max":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"last":0.247}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":421888,"data_stored":374997,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":765952,"data_stored":707823,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":491520,"data_stored":470398,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T11:08:55.363 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-16T11:08:55.363 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T11:08:55.619 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-16T11:08:55.619 INFO:teuthology.orchestra.run.smithi102.stderr:dumped all 2024-09-16T11:08:55.632 INFO:teuthology.orchestra.run.smithi102.stdout:{"pg_ready":true,"pg_map":{"version":71,"stamp":"2024-09-16T11:08:53.880660+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":38133882,"num_objects":63,"num_object_clones":19,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63,"num_whiteouts":7,"num_read":15710,"num_read_kb":48777,"num_write":3030,"num_write_kb":29004,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2401,"ondisk_log_size":2401,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":917432,"kb_used_data":4560,"kb_used_omap":4,"kb_used_meta":912827,"kb_avail":282198088,"statfs":{"total":289910292480,"available":288970842112,"internally_reserved":0,"allocated":4669440,"data_stored":4850886,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":5112,"internal_metadata":934734856},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-161246497,"num_objects":-114,"num_object_clones":-24,"num_object_copies":-228,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-114,"num_whiteouts":-19,"num_read":1251,"num_read_kb":1071,"num_write":242,"num_write_kb":63,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.021852"},"pg_stats":[{"pgid":"3.1f","version":"33'33","reported_seq":199,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-16T11:08:51.888152+0000","last_change":"2024-09-16T11:08:51.888351+0000","last_active":"2024-09-16T11:08:51.888152+0000","last_peered":"2024-09-16T11:08:51.888152+0000","last_clean":"2024-09-16T11:08:51.888152+0000","last_became_active":"2024-09-16T11:08:07.859102+0000","last_became_peered":"2024-09-16T11:08:07.859102+0000","last_unstale":"2024-09-16T11:08:51.888152+0000","last_undegraded":"2024-09-16T11:08:51.888152+0000","last_fullsized":"2024-09-16T11:08:51.888152+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:08:06.827173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:08:06.827173+0000","last_clean_scrub_stamp":"2024-09-16T11:08:06.827173+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":0,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:21:53.809689+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.008213639,"stat_sum":{"num_bytes":2813157,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":127,"num_read_kb":1006,"num_write":34,"num_write_kb":861,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"6","length":"5"}]},{"pgid":"3.1e","version":"33'155","reported_seq":741,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-16T11:08:52.349329+0000","last_change":"2024-09-16T11:08:51.877347+0000","last_active":"2024-09-16T11:08:52.349329+0000","last_peered":"2024-09-16T11:08:52.349329+0000","last_clean":"2024-09-16T11:08:52.349329+0000","last_became_active":"2024-09-16T11:08:07.856092+0000","last_became_peered":"2024-09-16T11:08:07.856092+0000","last_unstale":"2024-09-16T11:08:52.349329+0000","last_undegraded":"2024-09-16T11:08:52.349329+0000","last_fullsized":"2024-09-16T11:08:52.349329+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:08:06.827173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:08:06.827173+0000","last_clean_scrub_stamp":"2024-09-16T11:08:06.827173+0000","objects_scrubbed":0,"log_size":155,"log_dups_size":0,"ondisk_log_size":155,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:02:44.147643+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0023786520000000002,"stat_sum":{"num_bytes":37,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1807,"num_read_kb":2532,"num_write":196,"num_write_kb":424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"6","length":"5"}]},{"pgid":"3.1d","version":"28'275","reported_seq":332,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-16T11:08:51.877757+0000","last_change":"2024-09-16T11:08:51.878043+0000","last_active":"2024-09-16T11:08:51.877757+0000","last_peered":"2024-09-16T11:08:51.877757+0000","last_clean":"2024-09-16T11:08:51.877757+0000","last_became_active":"2024-09-16T11:08:07.847513+0000","last_became_peered":"2024-09-16T11:08:07.847513+0000","last_unstale":"2024-09-16T11:08:51.877757+0000","last_undegraded":"2024-09-16T11:08:51.877757+0000","last_fullsized":"2024-09-16T11:08:51.877757+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:08:06.827173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:08:06.827173+0000","last_clean_scrub_stamp":"2024-09-16T11:08:06.827173+0000","objects_scrubbed":0,"log_size":275,"log_dups_size":0,"ondisk_log_size":275,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:51:39.065833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.003055496,"stat_sum":{"num_bytes":42,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1234,"num_read_kb":1012,"num_write":445,"num_write_kb":597,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"6","length":"5"}]},{"pgid":"3.1c","version":"33'33","reported_seq":213,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-16T11:08:52.109073+0000","last_change":"2024-09-16T11:08:51.882421+0000","last_active":"2024-09-16T11:08:52.109073+0000","last_peered":"2024-09-16T11:08:52.109073+0000","last_clean":"2024-09-16T11:08:52.109073+0000","last_became_active":"2024-09-16T11:08:07.855430+0000","last_became_peered":"2024-09-16T11:08:07.855430+0000","last_unstale":"2024-09-16T11:08:52.109073+0000","last_undegraded":"2024-09-16T11:08:52.109073+0000","last_fullsized":"2024-09-16T11:08:52.109073+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:08:06.827173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:08:06.827173+0000","last_clean_scrub_stamp":"2024-09-16T11:08:06.827173+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":0,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:28:07.715262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0048504100000000003,"stat_sum":{"num_bytes":1869110,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":167,"num_read_kb":250,"num_write":42,"num_write_kb":566,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"6","length":"5"}]},{"pgid":"3.1b","version":"33'160","reported_seq":840,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-16T11:08:51.888964+0000","last_change":"2024-09-16T11:08:51.889166+0000","last_active":"2024-09-16T11:08:51.888964+0000","last_peered":"2024-09-16T11:08:51.888964+0000","last_clean":"2024-09-16T11:08:51.888964+0000","last_became_active":"2024-09-16T11:08:07.856827+0000","last_became_peered":"2024-09-16T11:08:07.856827+0000","last_unstale":"2024-09-16T11:08:51.888964+0000","last_undegraded":"2024-09-16T11:08:51.888964+0000","last_fullsized":"2024-09-16T11:08:51.888964+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:08:06.827173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:08:06.827173+0000","last_clean_scrub_stamp":"2024-09-16T11:08:06.827173+0000","objects_scrubbed":0,"log_size":160,"log_dups_size":0,"ondisk_log_size":160,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:47:27.295935+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0088276499999999994,"stat_sum":{"num_bytes":590386,"num_objects":4,"num_object_clones":2,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":1,"num_read":1407,"num_read_kb":2244,"num_write":139,"num_write_kb":887,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"6","length":"5"}]},{"pgid":"3.1a","version":"28'13","reported_seq":374,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-16T11:08:52.316726+0000","last_change":"2024-09-16T11:08:51.887798+0000","last_active":"2024-09-16T11:08:52.316726+0000","last_peered":"2024-09-16T11:08:52.316726+0000","last_clean":"2024-09-16T11:08:52.316726+0000","last_became_active":"2024-09-16T11:08:07.853372+0000","last_became_peered":"2024-09-16T11:08:07.853372+0000","last_unstale":"2024-09-16T11:08:52.316726+0000","last_undegraded":"2024-09-16T11:08:52.316726+0000","last_fullsized":"2024-09-16T11:08:52.316726+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:08:06.827173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:08:06.827173+0000","last_clean_scrub_stamp":"2024-09-16T11:08:06.827173+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:21:53.809689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0089025300000000005,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":315,"num_read_kb":333,"num_write":13,"num_write_kb":81,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"6","length":"5"}]},{"pgid":"3.19","version":"32'39","reported_seq":280,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-16T11:08:51.881126+0000","last_change":"2024-09-16T11:08:51.881384+0000","last_active":"2024-09-16T11:08:51.881126+0000","last_peered":"2024-09-16T11:08:51.881126+0000","last_clean":"2024-09-16T11:08:51.881126+0000","last_became_active":"2024-09-16T11:08:07.852874+0000","last_became_peered":"2024-09-16T11:08:07.852874+0000","last_unstale":"2024-09-16T11:08:51.881126+0000","last_undegraded":"2024-09-16T11:08:51.881126+0000","last_fullsized":"2024-09-16T11:08:51.881126+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:08:06.827173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:08:06.827173+0000","last_clean_scrub_stamp":"2024-09-16T11:08:06.827173+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:47:27.295935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0042733110000000001,"stat_sum":{"num_bytes":6202271,"num_objects":6,"num_object_clones":2,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":214,"num_read_kb":667,"num_write":42,"num_write_kb":746,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"6","length":"5"}]},{"pgid":"3.18","version":"28'17","reported_seq":66,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-16T11:08:51.874889+0000","last_change":"2024-09-16T11:08:51.875249+0000","last_active":"2024-09-16T11:08:51.874889+0000","last_peered":"2024-09-16T11:08:51.874889+0000","last_clean":"2024-09-16T11:08:51.874889+0000","last_became_active":"2024-09-16T11:08:07.848434+0000","last_became_peered":"2024-09-16T11:08:07.848434+0000","last_unstale":"2024-09-16T11:08:51.874889+0000","last_undegraded":"2024-09-16T11:08:51.874889+0000","last_fullsized":"2024-09-16T11:08:51.874889+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:08:06.827173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:08:06.827173+0000","last_clean_scrub_stamp":"2024-09-16T11:08:06.827173+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:47:04.478053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055608000000000003,"stat_sum":{"num_bytes":2342210,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":17,"num_write_kb":60,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"6","length":"5"}]},{"pgid":"3.17","version":"31'25","reported_seq":99,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-16T11:08:51.884747+0000","last_change":"2024-09-16T11:08:51.885023+0000","last_active":"2024-09-16T11:08:51.884747+0000","last_peered":"2024-09-16T11:08:51.884747+0000","last_clean":"2024-09-16T11:08:51.884747+0000","last_became_active":"2024-09-16T11:08:07.859097+0000","last_became_peered":"2024-09-16T11:08:07.859097+0000","last_unstale":"2024-09-16T11:08:51.884747+0000","last_undegraded":"2024-09-16T11:08:51.884747+0000","last_fullsized":"2024-09-16T11:08:51.884747+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:08:06.827173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:08:06.827173+0000","last_clean_scrub_stamp":"2024-09-16T11:08:06.827173+0000","objects_scrubbed":0,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:33:25.601273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0048877390000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":30,"num_read_kb":66,"num_write":24,"num_write_kb":325,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"6","length":"5"}]},{"pgid":"3.16","version":"32'25","reported_seq":128,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-16T11:08:51.878256+0000","last_change":"2024-09-16T11:08:51.878537+0000","last_active":"2024-09-16T11:08:51.878256+0000","last_peered":"2024-09-16T11:08:51.878256+0000","last_clean":"2024-09-16T11:08:51.878256+0000","last_became_active":"2024-09-16T11:08:07.856007+0000","last_became_peered":"2024-09-16T11:08:07.856007+0000","last_unstale":"2024-09-16T11:08:51.878256+0000","last_undegraded":"2024-09-16T11:08:51.878256+0000","last_fullsized":"2024-09-16T11:08:51.878256+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:08:06.827173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:08:06.827173+0000","last_clean_scrub_stamp":"2024-09-16T11:08:06.827173+0000","objects_scrubbed":0,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:18:30.121459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.003443495,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":66,"num_read_kb":1713,"num_write":38,"num_write_kb":1539,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"6","length":"5"}]},{"pgid":"3.15","version":"33'31","reported_seq":160,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-16T11:08:51.884698+0000","last_change":"2024-09-16T11:08:51.884698+0000","last_active":"2024-09-16T11:08:51.884698+0000","last_peered":"2024-09-16T11:08:51.884698+0000","last_clean":"2024-09-16T11:08:51.884698+0000","last_became_active":"2024-09-16T11:08:07.858375+0000","last_became_peered":"2024-09-16T11:08:07.858375+0000","last_unstale":"2024-09-16T11:08:51.884698+0000","last_undegraded":"2024-09-16T11:08:51.884698+0000","last_fullsized":"2024-09-16T11:08:51.884698+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:08:06.827173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:08:06.827173+0000","last_clean_scrub_stamp":"2024-09-16T11:08:06.827173+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:46:06.267425+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.0047413910000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":92,"num_read_kb":770,"num_write":36,"num_write_kb":1093,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"6","length":"5"}]},{"pgid":"3.14","version":"32'137","reported_seq":289,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-16T11:08:51.882715+0000","last_change":"2024-09-16T11:08:51.883437+0000","last_active":"2024-09-16T11:08:51.882715+0000","last_peered":"2024-09-16T11:08:51.882715+0000","last_clean":"2024-09-16T11:08:51.882715+0000","last_became_active":"2024-09-16T11:08:07.853189+0000","last_became_peered":"2024-09-16T11:08:07.853189+0000","last_unstale":"2024-09-16T11:08:51.882715+0000","last_undegraded":"2024-09-16T11:08:51.882715+0000","last_fullsized":"2024-09-16T11:08:51.882715+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:08:06.827173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:08:06.827173+0000","last_clean_scrub_stamp":"2024-09-16T11:08:06.827173+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:30:04.184361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0057208049999999998,"stat_sum":{"num_bytes":1600291,"num_objects":3,"num_object_clones":1,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":523,"num_read_kb":930,"num_write":187,"num_write_kb":943,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"6","length":"5"}]},{"pgid":"2.7","version":"0'0","reported_seq":33,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-16T11:08:50.865069+0000","last_change":"2024-09-16T11:07:46.454524+0000","last_active":"2024-09-16T11:08:50.865069+0000","last_peered":"2024-09-16T11:08:50.865069+0000","last_clean":"2024-09-16T11:08:50.865069+0000","last_became_active":"2024-09-16T11:07:44.446646+0000","last_became_peered":"2024-09-16T11:07:44.446646+0000","last_unstale":"2024-09-16T11:08:50.865069+0000","last_undegraded":"2024-09-16T11:08:50.865069+0000","last_fullsized":"2024-09-16T11:08:50.865069+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:07:43.427920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:07:43.427920+0000","last_clean_scrub_stamp":"2024-09-16T11:07:43.427920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:34:38.920978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000337344,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.6","version":"33'80","reported_seq":135,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-16T11:08:51.886576+0000","last_change":"2024-09-16T11:08:51.886576+0000","last_active":"2024-09-16T11:08:51.886576+0000","last_peered":"2024-09-16T11:08:51.886576+0000","last_clean":"2024-09-16T11:08:51.886576+0000","last_became_active":"2024-09-16T11:08:07.858208+0000","last_became_peered":"2024-09-16T11:08:07.858208+0000","last_unstale":"2024-09-16T11:08:51.886576+0000","last_undegraded":"2024-09-16T11:08:51.886576+0000","last_fullsized":"2024-09-16T11:08:51.886576+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:08:06.827173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:08:06.827173+0000","last_clean_scrub_stamp":"2024-09-16T11:08:06.827173+0000","objects_scrubbed":0,"log_size":80,"log_dups_size":0,"ondisk_log_size":80,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:20:16.980280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0063318970000000004,"stat_sum":{"num_bytes":3631436,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":259,"num_read_kb":213,"num_write":113,"num_write_kb":675,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"6","length":"5"}]},{"pgid":"2.6","version":"0'0","reported_seq":33,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-16T11:08:50.864872+0000","last_change":"2024-09-16T11:07:46.454822+0000","last_active":"2024-09-16T11:08:50.864872+0000","last_peered":"2024-09-16T11:08:50.864872+0000","last_clean":"2024-09-16T11:08:50.864872+0000","last_became_active":"2024-09-16T11:07:44.446406+0000","last_became_peered":"2024-09-16T11:07:44.446406+0000","last_unstale":"2024-09-16T11:08:50.864872+0000","last_undegraded":"2024-09-16T11:08:50.864872+0000","last_fullsized":"2024-09-16T11:08:50.864872+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:07:43.427920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:07:43.427920+0000","last_clean_scrub_stamp":"2024-09-16T11:07:43.427920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:12:45.650225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062301399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"31'41","reported_seq":314,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-16T11:08:51.878097+0000","last_change":"2024-09-16T11:08:51.878390+0000","last_active":"2024-09-16T11:08:51.878097+0000","last_peered":"2024-09-16T11:08:51.878097+0000","last_clean":"2024-09-16T11:08:51.878097+0000","last_became_active":"2024-09-16T11:08:07.856393+0000","last_became_peered":"2024-09-16T11:08:07.856393+0000","last_unstale":"2024-09-16T11:08:51.878097+0000","last_undegraded":"2024-09-16T11:08:51.878097+0000","last_fullsized":"2024-09-16T11:08:51.878097+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:08:06.827173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:08:06.827173+0000","last_clean_scrub_stamp":"2024-09-16T11:08:06.827173+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:11:35.974778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.003312602,"stat_sum":{"num_bytes":4131275,"num_objects":3,"num_object_clones":1,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":1,"num_read":241,"num_read_kb":2845,"num_write":54,"num_write_kb":2044,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"6","length":"5"}]},{"pgid":"2.5","version":"0'0","reported_seq":33,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-16T11:08:50.864449+0000","last_change":"2024-09-16T11:07:46.455055+0000","last_active":"2024-09-16T11:08:50.864449+0000","last_peered":"2024-09-16T11:08:50.864449+0000","last_clean":"2024-09-16T11:08:50.864449+0000","last_became_active":"2024-09-16T11:07:44.446498+0000","last_became_peered":"2024-09-16T11:07:44.446498+0000","last_unstale":"2024-09-16T11:08:50.864449+0000","last_undegraded":"2024-09-16T11:08:50.864449+0000","last_fullsized":"2024-09-16T11:08:50.864449+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:07:43.427920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:07:43.427920+0000","last_clean_scrub_stamp":"2024-09-16T11:07:43.427920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:40:08.726582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00084251400000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"32'67","reported_seq":266,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-16T11:08:51.883487+0000","last_change":"2024-09-16T11:08:51.883852+0000","last_active":"2024-09-16T11:08:51.883487+0000","last_peered":"2024-09-16T11:08:51.883487+0000","last_clean":"2024-09-16T11:08:51.883487+0000","last_became_active":"2024-09-16T11:08:07.853470+0000","last_became_peered":"2024-09-16T11:08:07.853470+0000","last_unstale":"2024-09-16T11:08:51.883487+0000","last_undegraded":"2024-09-16T11:08:51.883487+0000","last_fullsized":"2024-09-16T11:08:51.883487+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:08:06.827173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:08:06.827173+0000","last_clean_scrub_stamp":"2024-09-16T11:08:06.827173+0000","objects_scrubbed":0,"log_size":67,"log_dups_size":0,"ondisk_log_size":67,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:46:06.267425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0059983930000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":360,"num_read_kb":1468,"num_write":100,"num_write_kb":888,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"6","length":"5"}]},{"pgid":"2.4","version":"0'0","reported_seq":33,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-16T11:08:50.864582+0000","last_change":"2024-09-16T11:07:46.454774+0000","last_active":"2024-09-16T11:08:50.864582+0000","last_peered":"2024-09-16T11:08:50.864582+0000","last_clean":"2024-09-16T11:08:50.864582+0000","last_became_active":"2024-09-16T11:07:44.446413+0000","last_became_peered":"2024-09-16T11:07:44.446413+0000","last_unstale":"2024-09-16T11:08:50.864582+0000","last_undegraded":"2024-09-16T11:08:50.864582+0000","last_fullsized":"2024-09-16T11:08:50.864582+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:07:43.427920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:07:43.427920+0000","last_clean_scrub_stamp":"2024-09-16T11:07:43.427920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:53:07.204424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059042800000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"32'69","reported_seq":469,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-16T11:08:51.878615+0000","last_change":"2024-09-16T11:08:51.879403+0000","last_active":"2024-09-16T11:08:51.878615+0000","last_peered":"2024-09-16T11:08:51.878615+0000","last_clean":"2024-09-16T11:08:51.878615+0000","last_became_active":"2024-09-16T11:08:07.855280+0000","last_became_peered":"2024-09-16T11:08:07.855280+0000","last_unstale":"2024-09-16T11:08:51.878615+0000","last_undegraded":"2024-09-16T11:08:51.878615+0000","last_fullsized":"2024-09-16T11:08:51.878615+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:08:06.827173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:08:06.827173+0000","last_clean_scrub_stamp":"2024-09-16T11:08:06.827173+0000","objects_scrubbed":0,"log_size":69,"log_dups_size":0,"ondisk_log_size":69,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:03:42.315998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0042484180000000003,"stat_sum":{"num_bytes":786432,"num_objects":6,"num_object_clones":3,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":396,"num_read_kb":2349,"num_write":76,"num_write_kb":1069,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"6","length":"5"}]},{"pgid":"3.3","version":"33'78","reported_seq":247,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-16T11:08:52.210563+0000","last_change":"2024-09-16T11:08:51.886868+0000","last_active":"2024-09-16T11:08:52.210563+0000","last_peered":"2024-09-16T11:08:52.210563+0000","last_clean":"2024-09-16T11:08:52.210563+0000","last_became_active":"2024-09-16T11:08:07.847550+0000","last_became_peered":"2024-09-16T11:08:07.847550+0000","last_unstale":"2024-09-16T11:08:52.210563+0000","last_undegraded":"2024-09-16T11:08:52.210563+0000","last_fullsized":"2024-09-16T11:08:52.210563+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:08:06.827173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:08:06.827173+0000","last_clean_scrub_stamp":"2024-09-16T11:08:06.827173+0000","objects_scrubbed":0,"log_size":78,"log_dups_size":0,"ondisk_log_size":78,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:25:26.863744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0065329400000000001,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":152,"num_read_kb":956,"num_write":108,"num_write_kb":2573,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"6","length":"5"}]},{"pgid":"2.2","version":"15'2","reported_seq":34,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-16T11:08:50.869072+0000","last_change":"2024-09-16T11:07:46.448909+0000","last_active":"2024-09-16T11:08:50.869072+0000","last_peered":"2024-09-16T11:08:50.869072+0000","last_clean":"2024-09-16T11:08:50.869072+0000","last_became_active":"2024-09-16T11:07:44.449069+0000","last_became_peered":"2024-09-16T11:07:44.449069+0000","last_unstale":"2024-09-16T11:08:50.869072+0000","last_undegraded":"2024-09-16T11:08:50.869072+0000","last_fullsized":"2024-09-16T11:08:50.869072+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:07:43.427920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:07:43.427920+0000","last_clean_scrub_stamp":"2024-09-16T11:07:43.427920+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:13:44.990400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00029305100000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.0","version":"33'46","reported_seq":203,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-16T11:08:52.184769+0000","last_change":"2024-09-16T11:08:51.881057+0000","last_active":"2024-09-16T11:08:52.184769+0000","last_peered":"2024-09-16T11:08:52.184769+0000","last_clean":"2024-09-16T11:08:52.184769+0000","last_became_active":"2024-09-16T11:08:07.855258+0000","last_became_peered":"2024-09-16T11:08:07.855258+0000","last_unstale":"2024-09-16T11:08:52.184769+0000","last_undegraded":"2024-09-16T11:08:52.184769+0000","last_fullsized":"2024-09-16T11:08:52.184769+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:08:06.827173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:08:06.827173+0000","last_clean_scrub_stamp":"2024-09-16T11:08:06.827173+0000","objects_scrubbed":0,"log_size":46,"log_dups_size":0,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:26:07.701673+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.0040037609999999998,"stat_sum":{"num_bytes":42,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":139,"num_read_kb":1672,"num_write":52,"num_write_kb":1263,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"6","length":"5"}]},{"pgid":"2.1","version":"0'0","reported_seq":32,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-16T11:08:50.861810+0000","last_change":"2024-09-16T11:07:46.720289+0000","last_active":"2024-09-16T11:08:50.861810+0000","last_peered":"2024-09-16T11:08:50.861810+0000","last_clean":"2024-09-16T11:08:50.861810+0000","last_became_active":"2024-09-16T11:07:44.447533+0000","last_became_peered":"2024-09-16T11:07:44.447533+0000","last_unstale":"2024-09-16T11:08:50.861810+0000","last_undegraded":"2024-09-16T11:08:50.861810+0000","last_fullsized":"2024-09-16T11:08:50.861810+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:07:43.427920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:07:43.427920+0000","last_clean_scrub_stamp":"2024-09-16T11:07:43.427920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:13:44.990400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000230378,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"33'30","reported_seq":160,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-16T11:08:51.882164+0000","last_change":"2024-09-16T11:08:51.883251+0000","last_active":"2024-09-16T11:08:51.882164+0000","last_peered":"2024-09-16T11:08:51.882164+0000","last_clean":"2024-09-16T11:08:51.882164+0000","last_became_active":"2024-09-16T11:08:07.847955+0000","last_became_peered":"2024-09-16T11:08:07.847955+0000","last_unstale":"2024-09-16T11:08:51.882164+0000","last_undegraded":"2024-09-16T11:08:51.882164+0000","last_fullsized":"2024-09-16T11:08:51.882164+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:08:06.827173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:08:06.827173+0000","last_clean_scrub_stamp":"2024-09-16T11:08:06.827173+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:02:44.147643+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.003530334,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":87,"num_read_kb":1164,"num_write":31,"num_write_kb":740,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"6","length":"5"}]},{"pgid":"2.0","version":"0'0","reported_seq":32,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-16T11:08:50.861284+0000","last_change":"2024-09-16T11:07:46.720264+0000","last_active":"2024-09-16T11:08:50.861284+0000","last_peered":"2024-09-16T11:08:50.861284+0000","last_clean":"2024-09-16T11:08:50.861284+0000","last_became_active":"2024-09-16T11:07:44.448153+0000","last_became_peered":"2024-09-16T11:07:44.448153+0000","last_unstale":"2024-09-16T11:08:50.861284+0000","last_undegraded":"2024-09-16T11:08:50.861284+0000","last_fullsized":"2024-09-16T11:08:50.861284+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:07:43.427920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:07:43.427920+0000","last_clean_scrub_stamp":"2024-09-16T11:07:43.427920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:09:42.536129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00024792500000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"33'40","reported_seq":266,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-16T11:08:52.254010+0000","last_change":"2024-09-16T11:08:51.889257+0000","last_active":"2024-09-16T11:08:52.254010+0000","last_peered":"2024-09-16T11:08:52.254010+0000","last_clean":"2024-09-16T11:08:52.254010+0000","last_became_active":"2024-09-16T11:08:07.853307+0000","last_became_peered":"2024-09-16T11:08:07.853307+0000","last_unstale":"2024-09-16T11:08:52.254010+0000","last_undegraded":"2024-09-16T11:08:52.254010+0000","last_fullsized":"2024-09-16T11:08:52.254010+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:08:06.827173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:08:06.827173+0000","last_clean_scrub_stamp":"2024-09-16T11:08:06.827173+0000","objects_scrubbed":0,"log_size":40,"log_dups_size":0,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:33:25.601273+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.010809077,"stat_sum":{"num_bytes":1243751,"num_objects":5,"num_object_clones":2,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":1,"num_read":203,"num_read_kb":1989,"num_write":37,"num_write_kb":217,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"6","length":"5"}]},{"pgid":"2.3","version":"13'1","reported_seq":34,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-16T11:08:50.865131+0000","last_change":"2024-09-16T11:07:46.454516+0000","last_active":"2024-09-16T11:08:50.865131+0000","last_peered":"2024-09-16T11:08:50.865131+0000","last_clean":"2024-09-16T11:08:50.865131+0000","last_became_active":"2024-09-16T11:07:44.446367+0000","last_became_peered":"2024-09-16T11:07:44.446367+0000","last_unstale":"2024-09-16T11:08:50.865131+0000","last_undegraded":"2024-09-16T11:08:50.865131+0000","last_fullsized":"2024-09-16T11:08:50.865131+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:07:43.427920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:07:43.427920+0000","last_clean_scrub_stamp":"2024-09-16T11:07:43.427920+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:06:21.202391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00038771200000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"10'52","reported_seq":98,"reported_epoch":32,"state":"active+clean","last_fresh":"2024-09-16T11:08:50.864840+0000","last_change":"2024-09-16T11:07:39.420884+0000","last_active":"2024-09-16T11:08:50.864840+0000","last_peered":"2024-09-16T11:08:50.864840+0000","last_clean":"2024-09-16T11:08:50.864840+0000","last_became_active":"2024-09-16T11:07:39.420564+0000","last_became_peered":"2024-09-16T11:07:39.420564+0000","last_unstale":"2024-09-16T11:08:50.864840+0000","last_undegraded":"2024-09-16T11:08:50.864840+0000","last_fullsized":"2024-09-16T11:08:50.864840+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:07:38.411971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:07:38.411971+0000","last_clean_scrub_stamp":"2024-09-16T11:07:38.411971+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:52:50.808244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":56,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"33'41","reported_seq":448,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-16T11:08:51.877660+0000","last_change":"2024-09-16T11:08:51.877660+0000","last_active":"2024-09-16T11:08:51.877660+0000","last_peered":"2024-09-16T11:08:51.877660+0000","last_clean":"2024-09-16T11:08:51.877660+0000","last_became_active":"2024-09-16T11:08:07.857280+0000","last_became_peered":"2024-09-16T11:08:07.857280+0000","last_unstale":"2024-09-16T11:08:51.877660+0000","last_undegraded":"2024-09-16T11:08:51.877660+0000","last_fullsized":"2024-09-16T11:08:51.877660+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:08:06.827173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:08:06.827173+0000","last_clean_scrub_stamp":"2024-09-16T11:08:06.827173+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:25:26.863744+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.0028720669999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":381,"num_read_kb":3183,"num_write":40,"num_write_kb":936,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"6","length":"5"}]},{"pgid":"3.9","version":"28'17","reported_seq":65,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-16T11:08:51.879719+0000","last_change":"2024-09-16T11:08:51.880725+0000","last_active":"2024-09-16T11:08:51.879719+0000","last_peered":"2024-09-16T11:08:51.879719+0000","last_clean":"2024-09-16T11:08:51.879719+0000","last_became_active":"2024-09-16T11:08:07.850403+0000","last_became_peered":"2024-09-16T11:08:07.850403+0000","last_unstale":"2024-09-16T11:08:51.879719+0000","last_undegraded":"2024-09-16T11:08:51.879719+0000","last_fullsized":"2024-09-16T11:08:51.879719+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:08:06.827173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:08:06.827173+0000","last_clean_scrub_stamp":"2024-09-16T11:08:06.827173+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:32:06.761762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011094919999999999,"stat_sum":{"num_bytes":3063089,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":3,"num_read_kb":55,"num_write":20,"num_write_kb":629,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"6","length":"5"}]},{"pgid":"3.a","version":"33'48","reported_seq":361,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-16T11:08:51.886253+0000","last_change":"2024-09-16T11:08:51.886253+0000","last_active":"2024-09-16T11:08:51.886253+0000","last_peered":"2024-09-16T11:08:51.886253+0000","last_clean":"2024-09-16T11:08:51.886253+0000","last_became_active":"2024-09-16T11:08:07.857407+0000","last_became_peered":"2024-09-16T11:08:07.857407+0000","last_unstale":"2024-09-16T11:08:51.886253+0000","last_undegraded":"2024-09-16T11:08:51.886253+0000","last_fullsized":"2024-09-16T11:08:51.886253+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:08:06.827173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:08:06.827173+0000","last_clean_scrub_stamp":"2024-09-16T11:08:06.827173+0000","objects_scrubbed":0,"log_size":48,"log_dups_size":0,"ondisk_log_size":48,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:27:32.110109+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0061513410000000003,"stat_sum":{"num_bytes":2050910,"num_objects":3,"num_object_clones":1,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":1,"num_read":288,"num_read_kb":3214,"num_write":54,"num_write_kb":1438,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"6","length":"5"}]},{"pgid":"3.b","version":"33'87","reported_seq":809,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-16T11:08:52.352279+0000","last_change":"2024-09-16T11:08:51.886242+0000","last_active":"2024-09-16T11:08:52.352279+0000","last_peered":"2024-09-16T11:08:52.352279+0000","last_clean":"2024-09-16T11:08:52.352279+0000","last_became_active":"2024-09-16T11:08:07.855380+0000","last_became_peered":"2024-09-16T11:08:07.855380+0000","last_unstale":"2024-09-16T11:08:52.352279+0000","last_undegraded":"2024-09-16T11:08:52.352279+0000","last_fullsized":"2024-09-16T11:08:52.352279+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:08:06.827173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:08:06.827173+0000","last_clean_scrub_stamp":"2024-09-16T11:08:06.827173+0000","objects_scrubbed":0,"log_size":87,"log_dups_size":0,"ondisk_log_size":87,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:48:03.017466+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0081692740000000007,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1188,"num_read_kb":4344,"num_write":97,"num_write_kb":1672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"6","length":"5"}]},{"pgid":"3.c","version":"32'70","reported_seq":398,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-16T11:08:51.881541+0000","last_change":"2024-09-16T11:08:51.881946+0000","last_active":"2024-09-16T11:08:51.881541+0000","last_peered":"2024-09-16T11:08:51.881541+0000","last_clean":"2024-09-16T11:08:51.881541+0000","last_became_active":"2024-09-16T11:08:07.859197+0000","last_became_peered":"2024-09-16T11:08:07.859197+0000","last_unstale":"2024-09-16T11:08:51.881541+0000","last_undegraded":"2024-09-16T11:08:51.881541+0000","last_fullsized":"2024-09-16T11:08:51.881541+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:08:06.827173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:08:06.827173+0000","last_clean_scrub_stamp":"2024-09-16T11:08:06.827173+0000","objects_scrubbed":0,"log_size":70,"log_dups_size":0,"ondisk_log_size":70,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:11:35.974778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019992830000000001,"stat_sum":{"num_bytes":3853000,"num_objects":3,"num_object_clones":1,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":1,"num_read":617,"num_read_kb":1509,"num_write":85,"num_write_kb":959,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"6","length":"5"}]},{"pgid":"3.d","version":"33'75","reported_seq":455,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-16T11:08:51.886372+0000","last_change":"2024-09-16T11:08:51.887845+0000","last_active":"2024-09-16T11:08:51.886372+0000","last_peered":"2024-09-16T11:08:51.886372+0000","last_clean":"2024-09-16T11:08:51.886372+0000","last_became_active":"2024-09-16T11:08:07.853865+0000","last_became_peered":"2024-09-16T11:08:07.853865+0000","last_unstale":"2024-09-16T11:08:51.886372+0000","last_undegraded":"2024-09-16T11:08:51.886372+0000","last_fullsized":"2024-09-16T11:08:51.886372+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:08:06.827173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:08:06.827173+0000","last_clean_scrub_stamp":"2024-09-16T11:08:06.827173+0000","objects_scrubbed":0,"log_size":75,"log_dups_size":0,"ondisk_log_size":75,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:35:17.633354+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0089916990000000006,"stat_sum":{"num_bytes":131072,"num_objects":2,"num_object_clones":1,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":785,"num_read_kb":1332,"num_write":69,"num_write_kb":661,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"6","length":"5"}]},{"pgid":"3.e","version":"33'52","reported_seq":312,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-16T11:08:51.881952+0000","last_change":"2024-09-16T11:08:51.881952+0000","last_active":"2024-09-16T11:08:51.881952+0000","last_peered":"2024-09-16T11:08:51.881952+0000","last_clean":"2024-09-16T11:08:51.881952+0000","last_became_active":"2024-09-16T11:08:07.847486+0000","last_became_peered":"2024-09-16T11:08:07.847486+0000","last_unstale":"2024-09-16T11:08:51.881952+0000","last_undegraded":"2024-09-16T11:08:51.881952+0000","last_fullsized":"2024-09-16T11:08:51.881952+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:08:06.827173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:08:06.827173+0000","last_clean_scrub_stamp":"2024-09-16T11:08:06.827173+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:40:57.767491+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.0067812660000000002,"stat_sum":{"num_bytes":196608,"num_objects":2,"num_object_clones":1,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":234,"num_read_kb":2061,"num_write":48,"num_write_kb":758,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"6","length":"5"}]},{"pgid":"3.f","version":"29'29","reported_seq":220,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-16T11:08:51.888411+0000","last_change":"2024-09-16T11:08:51.888577+0000","last_active":"2024-09-16T11:08:51.888411+0000","last_peered":"2024-09-16T11:08:51.888411+0000","last_clean":"2024-09-16T11:08:51.888411+0000","last_became_active":"2024-09-16T11:08:07.859480+0000","last_became_peered":"2024-09-16T11:08:07.859480+0000","last_unstale":"2024-09-16T11:08:51.888411+0000","last_undegraded":"2024-09-16T11:08:51.888411+0000","last_fullsized":"2024-09-16T11:08:51.888411+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:08:06.827173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:08:06.827173+0000","last_clean_scrub_stamp":"2024-09-16T11:08:06.827173+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:47:04.478053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0077316599999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":155,"num_read_kb":866,"num_write":30,"num_write_kb":312,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"6","length":"5"}]},{"pgid":"3.10","version":"33'33","reported_seq":160,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-16T11:08:51.882406+0000","last_change":"2024-09-16T11:08:51.882679+0000","last_active":"2024-09-16T11:08:51.882406+0000","last_peered":"2024-09-16T11:08:51.882406+0000","last_clean":"2024-09-16T11:08:51.882406+0000","last_became_active":"2024-09-16T11:08:07.855591+0000","last_became_peered":"2024-09-16T11:08:07.855591+0000","last_unstale":"2024-09-16T11:08:51.882406+0000","last_undegraded":"2024-09-16T11:08:51.882406+0000","last_fullsized":"2024-09-16T11:08:51.882406+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:08:06.827173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:08:06.827173+0000","last_clean_scrub_stamp":"2024-09-16T11:08:06.827173+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":0,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:20:16.980280+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0057037219999999996,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":83,"num_read_kb":512,"num_write":38,"num_write_kb":467,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"6","length":"5"}]},{"pgid":"3.11","version":"33'88","reported_seq":947,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-16T11:08:52.203437+0000","last_change":"2024-09-16T11:08:51.878317+0000","last_active":"2024-09-16T11:08:52.203437+0000","last_peered":"2024-09-16T11:08:52.203437+0000","last_clean":"2024-09-16T11:08:52.203437+0000","last_became_active":"2024-09-16T11:08:07.856626+0000","last_became_peered":"2024-09-16T11:08:07.856626+0000","last_unstale":"2024-09-16T11:08:52.203437+0000","last_undegraded":"2024-09-16T11:08:52.203437+0000","last_fullsized":"2024-09-16T11:08:52.203437+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:08:06.827173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:08:06.827173+0000","last_clean_scrub_stamp":"2024-09-16T11:08:06.827173+0000","objects_scrubbed":0,"log_size":88,"log_dups_size":0,"ondisk_log_size":88,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:27:32.110109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0031249049999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1527,"num_read_kb":3994,"num_write":85,"num_write_kb":292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"6","length":"5"}]},{"pgid":"3.12","version":"33'184","reported_seq":362,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-16T11:08:52.354747+0000","last_change":"2024-09-16T11:08:51.881467+0000","last_active":"2024-09-16T11:08:52.354747+0000","last_peered":"2024-09-16T11:08:52.354747+0000","last_clean":"2024-09-16T11:08:52.354747+0000","last_became_active":"2024-09-16T11:08:07.857698+0000","last_became_peered":"2024-09-16T11:08:07.857698+0000","last_unstale":"2024-09-16T11:08:52.354747+0000","last_undegraded":"2024-09-16T11:08:52.354747+0000","last_fullsized":"2024-09-16T11:08:52.354747+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:08:06.827173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:08:06.827173+0000","last_clean_scrub_stamp":"2024-09-16T11:08:06.827173+0000","objects_scrubbed":0,"log_size":184,"log_dups_size":0,"ondisk_log_size":184,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:30:04.184361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0016087269999999999,"stat_sum":{"num_bytes":327680,"num_objects":4,"num_object_clones":2,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":912,"num_read_kb":991,"num_write":286,"num_write_kb":407,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"6","length":"5"}]},{"pgid":"3.13","version":"32'228","reported_seq":679,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-16T11:08:51.881474+0000","last_change":"2024-09-16T11:08:51.881853+0000","last_active":"2024-09-16T11:08:51.881474+0000","last_peered":"2024-09-16T11:08:51.881474+0000","last_clean":"2024-09-16T11:08:51.881474+0000","last_became_active":"2024-09-16T11:08:07.853749+0000","last_became_peered":"2024-09-16T11:08:07.853749+0000","last_unstale":"2024-09-16T11:08:51.881474+0000","last_undegraded":"2024-09-16T11:08:51.881474+0000","last_fullsized":"2024-09-16T11:08:51.881474+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:08:06.827173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:08:06.827173+0000","last_clean_scrub_stamp":"2024-09-16T11:08:06.827173+0000","objects_scrubbed":0,"log_size":228,"log_dups_size":0,"ondisk_log_size":228,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:24:10.327154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0045125729999999998,"stat_sum":{"num_bytes":2710677,"num_objects":4,"num_object_clones":2,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":2,"num_read":1659,"num_read_kb":2488,"num_write":313,"num_write_kb":1774,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"6","length":"5"}]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":37543495,"num_objects":59,"num_object_clones":19,"num_object_copies":118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59,"num_whiteouts":7,"num_read":15654,"num_read_kb":48731,"num_write":2949,"num_write_kb":27896,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1679360,"data_stored":1553218,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2346,"ondisk_log_size":2346,"up":64,"acting":64,"num_store_stats":3},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":56,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":52,"ondisk_log_size":52,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359738387,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302260,"kb_used_data":748,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94069580,"statfs":{"total":96636764160,"available":96327249920,"internally_reserved":0,"allocated":765952,"data_stored":576358,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 11:08:40 2024","interfaces":[{"interface":"back","average":{"1min":0.221,"5min":0.221,"15min":0.221},"min":{"1min":0.14599999999999999,"5min":0.14599999999999999,"15min":0.14599999999999999},"max":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"last":0.28299999999999997},{"interface":"front","average":{"1min":0.23200000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"min":{"1min":0.17000000000000001,"5min":0.17000000000000001,"15min":0.17000000000000001},"max":{"1min":0.31,"5min":0.31,"15min":0.31},"last":0.24199999999999999}]},{"osd":1,"last update":"Mon Sep 16 11:08:40 2024","interfaces":[{"interface":"back","average":{"1min":0.23799999999999999,"5min":0.23799999999999999,"15min":0.23799999999999999},"min":{"1min":0.159,"5min":0.159,"15min":0.159},"max":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"last":0.22800000000000001},{"interface":"front","average":{"1min":0.252,"5min":0.252,"15min":0.252},"min":{"1min":0.184,"5min":0.184,"15min":0.184},"max":{"1min":0.376,"5min":0.376,"15min":0.376},"last":0.253}]}]},{"osd":1,"up_from":8,"seq":34359738388,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":308228,"kb_used_data":2548,"kb_used_omap":1,"kb_used_meta":305662,"kb_avail":94063612,"statfs":{"total":96636764160,"available":96321138688,"internally_reserved":0,"allocated":2609152,"data_stored":3203184,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":312998232},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 11:08:37 2024","interfaces":[{"interface":"back","average":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"min":{"1min":0.17399999999999999,"5min":0.17399999999999999,"15min":0.17399999999999999},"max":{"1min":0.376,"5min":0.376,"15min":0.376},"last":0.193},{"interface":"front","average":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"min":{"1min":0.17599999999999999,"5min":0.17599999999999999,"15min":0.17599999999999999},"max":{"1min":0.376,"5min":0.376,"15min":0.376},"last":0.20300000000000001}]},{"osd":2,"last update":"Mon Sep 16 11:08:37 2024","interfaces":[{"interface":"back","average":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"min":{"1min":0.16600000000000001,"5min":0.16600000000000001,"15min":0.16600000000000001},"max":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"last":0.183},{"interface":"front","average":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"min":{"1min":0.191,"5min":0.191,"15min":0.191},"max":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"last":0.217}]}]},{"osd":0,"up_from":8,"seq":34359738387,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":306944,"kb_used_data":1264,"kb_used_omap":1,"kb_used_meta":305662,"kb_avail":94064896,"statfs":{"total":96636764160,"available":96322453504,"internally_reserved":0,"allocated":1294336,"data_stored":1071344,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":312998232},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Sep 16 11:08:40 2024","interfaces":[{"interface":"back","average":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"min":{"1min":0.14599999999999999,"5min":0.14599999999999999,"15min":0.14599999999999999},"max":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"last":0.17899999999999999},{"interface":"front","average":{"1min":0.31,"5min":0.31,"15min":0.31},"min":{"1min":0.16600000000000001,"5min":0.16600000000000001,"15min":0.16600000000000001},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.20899999999999999}]},{"osd":2,"last update":"Mon Sep 16 11:08:40 2024","interfaces":[{"interface":"back","average":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"min":{"1min":0.20300000000000001,"5min":0.20300000000000001,"15min":0.20300000000000001},"max":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"last":0.19500000000000001},{"interface":"front","average":{"1min":0.309,"5min":0.309,"15min":0.309},"min":{"1min":0.186,"5min":0.186,"15min":0.186},"max":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"last":0.247}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":421888,"data_stored":374997,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":765952,"data_stored":707823,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":491520,"data_stored":470398,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T11:08:55.634 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-16T11:08:55.896 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-16T11:08:55.896 INFO:teuthology.orchestra.run.smithi102.stdout:{"epoch":35,"fsid":"79f34890-6ac0-492b-826f-b29cf1045a98","created":"2024-09-16T11:07:27.825113+0000","modified":"2024-09-16T11:08:53.878423+0000","last_up_change":"2024-09-16T11:07:37.411085+0000","last_in_change":"2024-09-16T11:07:28.722914+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T11:07:37.676106+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"11","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2024-09-16T11:07:42.972213+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":15,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"pool_client.0","create_time":"2024-09-16T11:08:06.142273+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":17,"snap_epoch":35,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"73775a88-4914-46f2-a066-b4f012c226c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6800","nonce":32395},{"type":"v1","addr":"172.21.15.102:6801","nonce":32395}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6802","nonce":32395},{"type":"v1","addr":"172.21.15.102:6803","nonce":32395}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6806","nonce":32395},{"type":"v1","addr":"172.21.15.102:6807","nonce":32395}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6804","nonce":32395},{"type":"v1","addr":"172.21.15.102:6805","nonce":32395}]},"public_addr":"172.21.15.102:6801/32395","cluster_addr":"172.21.15.102:6803/32395","heartbeat_back_addr":"172.21.15.102:6807/32395","heartbeat_front_addr":"172.21.15.102:6805/32395","state":["exists","up"]},{"osd":1,"uuid":"566bbdc4-ed6a-4580-a04b-7d4fa97fcc81","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6816","nonce":32397},{"type":"v1","addr":"172.21.15.102:6817","nonce":32397}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6818","nonce":32397},{"type":"v1","addr":"172.21.15.102:6819","nonce":32397}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6822","nonce":32397},{"type":"v1","addr":"172.21.15.102:6823","nonce":32397}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6820","nonce":32397},{"type":"v1","addr":"172.21.15.102:6821","nonce":32397}]},"public_addr":"172.21.15.102:6817/32397","cluster_addr":"172.21.15.102:6819/32397","heartbeat_back_addr":"172.21.15.102:6823/32397","heartbeat_front_addr":"172.21.15.102:6821/32397","state":["exists","up"]},{"osd":2,"uuid":"4aaa7267-412c-4e0f-b6f9-dd70e45cdc8a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6808","nonce":32396},{"type":"v1","addr":"172.21.15.102:6809","nonce":32396}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6810","nonce":32396},{"type":"v1","addr":"172.21.15.102:6811","nonce":32396}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6814","nonce":32396},{"type":"v1","addr":"172.21.15.102:6815","nonce":32396}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.102:6812","nonce":32396},{"type":"v1","addr":"172.21.15.102:6813","nonce":32396}]},"public_addr":"172.21.15.102:6809/32396","cluster_addr":"172.21.15.102:6811/32396","heartbeat_back_addr":"172.21.15.102:6815/32396","heartbeat_front_addr":"172.21.15.102:6813/32396","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T11:07:33.892765+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T11:07:33.888914+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T11:07:33.917416+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":3,"snaps":[{"begin":4,"length":7}]}],"new_removed_snaps":[{"pool":3,"snaps":[{"begin":4,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T11:08:56.909 INFO:tasks.ceph:Scrubbing osd.0 2024-09-16T11:08:56.909 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-09-16T11:08:57.035 INFO:teuthology.orchestra.run.smithi102.stdout:{ 2024-09-16T11:08:57.036 INFO:teuthology.orchestra.run.smithi102.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-16T11:08:57.036 INFO:teuthology.orchestra.run.smithi102.stdout:} 2024-09-16T11:08:57.046 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-09-16T11:08:57.304 INFO:teuthology.orchestra.run.smithi102.stderr:instructed osd(s) 0 to deep-scrub 2024-09-16T11:08:57.317 INFO:tasks.ceph:Scrubbing osd.1 2024-09-16T11:08:57.317 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-09-16T11:08:57.430 INFO:teuthology.orchestra.run.smithi102.stdout:{ 2024-09-16T11:08:57.430 INFO:teuthology.orchestra.run.smithi102.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-16T11:08:57.430 INFO:teuthology.orchestra.run.smithi102.stdout:} 2024-09-16T11:08:57.440 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-09-16T11:08:57.698 INFO:teuthology.orchestra.run.smithi102.stderr:instructed osd(s) 1 to deep-scrub 2024-09-16T11:08:57.711 INFO:tasks.ceph:Scrubbing osd.2 2024-09-16T11:08:57.711 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-09-16T11:08:57.826 INFO:teuthology.orchestra.run.smithi102.stdout:{ 2024-09-16T11:08:57.826 INFO:teuthology.orchestra.run.smithi102.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-16T11:08:57.826 INFO:teuthology.orchestra.run.smithi102.stdout:} 2024-09-16T11:08:57.836 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-09-16T11:08:58.097 INFO:teuthology.orchestra.run.smithi102.stderr:instructed osd(s) 2 to deep-scrub 2024-09-16T11:08:58.108 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T11:08:58.365 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-16T11:08:58.366 INFO:teuthology.orchestra.run.smithi102.stderr:dumped all 2024-09-16T11:08:58.377 INFO:teuthology.orchestra.run.smithi102.stdout:{"pg_ready":true,"pg_map":{"version":74,"stamp":"2024-09-16T11:08:57.819478+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":16061,"num_read_kb":49084,"num_write":3122,"num_write_kb":29026,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2539,"ondisk_log_size":2539,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":914932,"kb_used_data":2032,"kb_used_omap":4,"kb_used_meta":912827,"kb_avail":282200588,"statfs":{"total":289910292480,"available":288973402112,"internally_reserved":0,"allocated":2080768,"data_stored":1522931,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":5112,"internal_metadata":934734856},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-198789973,"num_objects":-169,"num_object_clones":-43,"num_object_copies":-338,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-169,"num_whiteouts":-26,"num_read":1602,"num_read_kb":1378,"num_write":334,"num_write_kb":85,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.947559"},"pg_stats":[{"pgid":"3.1f","version":"35'35","reported_seq":214,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T11:08:54.185384+0000","last_change":"2024-09-16T11:08:53.899921+0000","last_active":"2024-09-16T11:08:54.185384+0000","last_peered":"2024-09-16T11:08:54.185384+0000","last_clean":"2024-09-16T11:08:54.185384+0000","last_became_active":"2024-09-16T11:08:07.859102+0000","last_became_peered":"2024-09-16T11:08:07.859102+0000","last_unstale":"2024-09-16T11:08:54.185384+0000","last_undegraded":"2024-09-16T11:08:54.185384+0000","last_fullsized":"2024-09-16T11:08:54.185384+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:08:06.827173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:08:06.827173+0000","last_clean_scrub_stamp":"2024-09-16T11:08:06.827173+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00095524,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":130,"num_read_kb":1007,"num_write":36,"num_write_kb":861,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"4","length":"7"}]},{"pgid":"3.1e","version":"35'187","reported_seq":853,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T11:08:54.205213+0000","last_change":"2024-09-16T11:08:53.888422+0000","last_active":"2024-09-16T11:08:54.205213+0000","last_peered":"2024-09-16T11:08:54.205213+0000","last_clean":"2024-09-16T11:08:54.205213+0000","last_became_active":"2024-09-16T11:08:07.856092+0000","last_became_peered":"2024-09-16T11:08:07.856092+0000","last_unstale":"2024-09-16T11:08:54.205213+0000","last_undegraded":"2024-09-16T11:08:54.205213+0000","last_fullsized":"2024-09-16T11:08:54.205213+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:08:06.827173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:08:06.827173+0000","last_clean_scrub_stamp":"2024-09-16T11:08:06.827173+0000","objects_scrubbed":0,"log_size":187,"log_dups_size":0,"ondisk_log_size":187,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:02:44.147643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001038806,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1989,"num_read_kb":2691,"num_write":225,"num_write_kb":434,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"4","length":"7"}]},{"pgid":"3.1d","version":"35'287","reported_seq":358,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T11:08:54.193798+0000","last_change":"2024-09-16T11:08:53.888037+0000","last_active":"2024-09-16T11:08:54.193798+0000","last_peered":"2024-09-16T11:08:54.193798+0000","last_clean":"2024-09-16T11:08:54.193798+0000","last_became_active":"2024-09-16T11:08:07.847513+0000","last_became_peered":"2024-09-16T11:08:07.847513+0000","last_unstale":"2024-09-16T11:08:54.193798+0000","last_undegraded":"2024-09-16T11:08:54.193798+0000","last_fullsized":"2024-09-16T11:08:54.193798+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:08:06.827173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:08:06.827173+0000","last_clean_scrub_stamp":"2024-09-16T11:08:06.827173+0000","objects_scrubbed":0,"log_size":287,"log_dups_size":0,"ondisk_log_size":287,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:51:39.065833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000667575,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1265,"num_read_kb":1036,"num_write":459,"num_write_kb":602,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"4","length":"7"}]},{"pgid":"3.1c","version":"35'36","reported_seq":232,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T11:08:54.107936+0000","last_change":"2024-09-16T11:08:53.883362+0000","last_active":"2024-09-16T11:08:54.107936+0000","last_peered":"2024-09-16T11:08:54.107936+0000","last_clean":"2024-09-16T11:08:54.107936+0000","last_became_active":"2024-09-16T11:08:07.855430+0000","last_became_peered":"2024-09-16T11:08:07.855430+0000","last_unstale":"2024-09-16T11:08:54.107936+0000","last_undegraded":"2024-09-16T11:08:54.107936+0000","last_fullsized":"2024-09-16T11:08:54.107936+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:08:06.827173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:08:06.827173+0000","last_clean_scrub_stamp":"2024-09-16T11:08:06.827173+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:28:07.715262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097256700000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":177,"num_read_kb":260,"num_write":47,"num_write_kb":566,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"4","length":"7"}]},{"pgid":"3.1b","version":"34'165","reported_seq":855,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T11:08:53.899200+0000","last_change":"2024-09-16T11:08:53.899200+0000","last_active":"2024-09-16T11:08:53.899200+0000","last_peered":"2024-09-16T11:08:53.899200+0000","last_clean":"2024-09-16T11:08:53.899200+0000","last_became_active":"2024-09-16T11:08:07.856827+0000","last_became_peered":"2024-09-16T11:08:07.856827+0000","last_unstale":"2024-09-16T11:08:53.899200+0000","last_undegraded":"2024-09-16T11:08:53.899200+0000","last_fullsized":"2024-09-16T11:08:53.899200+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:08:06.827173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:08:06.827173+0000","last_clean_scrub_stamp":"2024-09-16T11:08:06.827173+0000","objects_scrubbed":0,"log_size":165,"log_dups_size":0,"ondisk_log_size":165,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081398199999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1407,"num_read_kb":2244,"num_write":140,"num_write_kb":887,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"4","length":"7"}]},{"pgid":"3.1a","version":"28'13","reported_seq":394,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T11:08:54.077030+0000","last_change":"2024-09-16T11:08:53.883198+0000","last_active":"2024-09-16T11:08:54.077030+0000","last_peered":"2024-09-16T11:08:54.077030+0000","last_clean":"2024-09-16T11:08:54.077030+0000","last_became_active":"2024-09-16T11:08:07.853372+0000","last_became_peered":"2024-09-16T11:08:07.853372+0000","last_unstale":"2024-09-16T11:08:54.077030+0000","last_undegraded":"2024-09-16T11:08:54.077030+0000","last_fullsized":"2024-09-16T11:08:54.077030+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:08:06.827173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:08:06.827173+0000","last_clean_scrub_stamp":"2024-09-16T11:08:06.827173+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:21:53.809689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00082456000000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":325,"num_read_kb":343,"num_write":13,"num_write_kb":81,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"4","length":"7"}]},{"pgid":"3.19","version":"35'47","reported_seq":297,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T11:08:54.184356+0000","last_change":"2024-09-16T11:08:53.882678+0000","last_active":"2024-09-16T11:08:54.184356+0000","last_peered":"2024-09-16T11:08:54.184356+0000","last_clean":"2024-09-16T11:08:54.184356+0000","last_became_active":"2024-09-16T11:08:07.852874+0000","last_became_peered":"2024-09-16T11:08:07.852874+0000","last_unstale":"2024-09-16T11:08:54.184356+0000","last_undegraded":"2024-09-16T11:08:54.184356+0000","last_fullsized":"2024-09-16T11:08:54.184356+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:08:06.827173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:08:06.827173+0000","last_clean_scrub_stamp":"2024-09-16T11:08:06.827173+0000","objects_scrubbed":0,"log_size":47,"log_dups_size":0,"ondisk_log_size":47,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:47:27.295935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00052581399999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":214,"num_read_kb":667,"num_write":46,"num_write_kb":746,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"4","length":"7"}]},{"pgid":"3.18","version":"35'18","reported_seq":77,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T11:08:53.887745+0000","last_change":"2024-09-16T11:08:53.887745+0000","last_active":"2024-09-16T11:08:53.887745+0000","last_peered":"2024-09-16T11:08:53.887745+0000","last_clean":"2024-09-16T11:08:53.887745+0000","last_became_active":"2024-09-16T11:08:07.848434+0000","last_became_peered":"2024-09-16T11:08:07.848434+0000","last_unstale":"2024-09-16T11:08:53.887745+0000","last_undegraded":"2024-09-16T11:08:53.887745+0000","last_fullsized":"2024-09-16T11:08:53.887745+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:08:06.827173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:08:06.827173+0000","last_clean_scrub_stamp":"2024-09-16T11:08:06.827173+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:47:04.478053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043492900000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":18,"num_write_kb":60,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"4","length":"7"}]},{"pgid":"3.17","version":"31'25","reported_seq":110,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T11:08:53.900234+0000","last_change":"2024-09-16T11:08:53.900234+0000","last_active":"2024-09-16T11:08:53.900234+0000","last_peered":"2024-09-16T11:08:53.900234+0000","last_clean":"2024-09-16T11:08:53.900234+0000","last_became_active":"2024-09-16T11:08:07.859097+0000","last_became_peered":"2024-09-16T11:08:07.859097+0000","last_unstale":"2024-09-16T11:08:53.900234+0000","last_undegraded":"2024-09-16T11:08:53.900234+0000","last_fullsized":"2024-09-16T11:08:53.900234+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:08:06.827173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:08:06.827173+0000","last_clean_scrub_stamp":"2024-09-16T11:08:06.827173+0000","objects_scrubbed":0,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001161507,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":30,"num_read_kb":66,"num_write":24,"num_write_kb":325,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"4","length":"7"}]},{"pgid":"3.16","version":"32'25","reported_seq":138,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T11:08:53.888529+0000","last_change":"2024-09-16T11:08:53.888681+0000","last_active":"2024-09-16T11:08:53.888529+0000","last_peered":"2024-09-16T11:08:53.888529+0000","last_clean":"2024-09-16T11:08:53.888529+0000","last_became_active":"2024-09-16T11:08:07.856007+0000","last_became_peered":"2024-09-16T11:08:07.856007+0000","last_unstale":"2024-09-16T11:08:53.888529+0000","last_undegraded":"2024-09-16T11:08:53.888529+0000","last_fullsized":"2024-09-16T11:08:53.888529+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:08:06.827173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:08:06.827173+0000","last_clean_scrub_stamp":"2024-09-16T11:08:06.827173+0000","objects_scrubbed":0,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:18:30.121459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011111560000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":66,"num_read_kb":1713,"num_write":38,"num_write_kb":1539,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"4","length":"7"}]},{"pgid":"3.15","version":"33'31","reported_seq":171,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T11:08:53.899692+0000","last_change":"2024-09-16T11:08:53.899692+0000","last_active":"2024-09-16T11:08:53.899692+0000","last_peered":"2024-09-16T11:08:53.899692+0000","last_clean":"2024-09-16T11:08:53.899692+0000","last_became_active":"2024-09-16T11:08:07.858375+0000","last_became_peered":"2024-09-16T11:08:07.858375+0000","last_unstale":"2024-09-16T11:08:53.899692+0000","last_undegraded":"2024-09-16T11:08:53.899692+0000","last_fullsized":"2024-09-16T11:08:53.899692+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:08:06.827173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:08:06.827173+0000","last_clean_scrub_stamp":"2024-09-16T11:08:06.827173+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00079626899999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":92,"num_read_kb":770,"num_write":36,"num_write_kb":1093,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"4","length":"7"}]},{"pgid":"3.14","version":"35'141","reported_seq":303,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T11:08:53.883413+0000","last_change":"2024-09-16T11:08:53.883413+0000","last_active":"2024-09-16T11:08:53.883413+0000","last_peered":"2024-09-16T11:08:53.883413+0000","last_clean":"2024-09-16T11:08:53.883413+0000","last_became_active":"2024-09-16T11:08:07.853189+0000","last_became_peered":"2024-09-16T11:08:07.853189+0000","last_unstale":"2024-09-16T11:08:53.883413+0000","last_undegraded":"2024-09-16T11:08:53.883413+0000","last_fullsized":"2024-09-16T11:08:53.883413+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:08:06.827173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:08:06.827173+0000","last_clean_scrub_stamp":"2024-09-16T11:08:06.827173+0000","objects_scrubbed":0,"log_size":141,"log_dups_size":0,"ondisk_log_size":141,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:30:04.184361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087565099999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":523,"num_read_kb":930,"num_write":189,"num_write_kb":943,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"4","length":"7"}]},{"pgid":"2.7","version":"0'0","reported_seq":35,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T11:08:52.879401+0000","last_change":"2024-09-16T11:07:46.454524+0000","last_active":"2024-09-16T11:08:52.879401+0000","last_peered":"2024-09-16T11:08:52.879401+0000","last_clean":"2024-09-16T11:08:52.879401+0000","last_became_active":"2024-09-16T11:07:44.446646+0000","last_became_peered":"2024-09-16T11:07:44.446646+0000","last_unstale":"2024-09-16T11:08:52.879401+0000","last_undegraded":"2024-09-16T11:08:52.879401+0000","last_fullsized":"2024-09-16T11:08:52.879401+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:07:43.427920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:07:43.427920+0000","last_clean_scrub_stamp":"2024-09-16T11:07:43.427920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:34:38.920978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000337344,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.6","version":"35'83","reported_seq":149,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T11:08:54.210790+0000","last_change":"2024-09-16T11:08:53.899856+0000","last_active":"2024-09-16T11:08:54.210790+0000","last_peered":"2024-09-16T11:08:54.210790+0000","last_clean":"2024-09-16T11:08:54.210790+0000","last_became_active":"2024-09-16T11:08:07.858208+0000","last_became_peered":"2024-09-16T11:08:07.858208+0000","last_unstale":"2024-09-16T11:08:54.210790+0000","last_undegraded":"2024-09-16T11:08:54.210790+0000","last_fullsized":"2024-09-16T11:08:54.210790+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:08:06.827173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:08:06.827173+0000","last_clean_scrub_stamp":"2024-09-16T11:08:06.827173+0000","objects_scrubbed":0,"log_size":83,"log_dups_size":0,"ondisk_log_size":83,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098881699999999991,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":259,"num_read_kb":213,"num_write":114,"num_write_kb":675,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"4","length":"7"}]},{"pgid":"2.6","version":"0'0","reported_seq":35,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T11:08:52.878816+0000","last_change":"2024-09-16T11:07:46.454822+0000","last_active":"2024-09-16T11:08:52.878816+0000","last_peered":"2024-09-16T11:08:52.878816+0000","last_clean":"2024-09-16T11:08:52.878816+0000","last_became_active":"2024-09-16T11:07:44.446406+0000","last_became_peered":"2024-09-16T11:07:44.446406+0000","last_unstale":"2024-09-16T11:08:52.878816+0000","last_undegraded":"2024-09-16T11:08:52.878816+0000","last_fullsized":"2024-09-16T11:08:52.878816+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:07:43.427920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:07:43.427920+0000","last_clean_scrub_stamp":"2024-09-16T11:07:43.427920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:12:45.650225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062301399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"35'44","reported_seq":327,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T11:08:53.887995+0000","last_change":"2024-09-16T11:08:53.887995+0000","last_active":"2024-09-16T11:08:53.887995+0000","last_peered":"2024-09-16T11:08:53.887995+0000","last_clean":"2024-09-16T11:08:53.887995+0000","last_became_active":"2024-09-16T11:08:07.856393+0000","last_became_peered":"2024-09-16T11:08:07.856393+0000","last_unstale":"2024-09-16T11:08:53.887995+0000","last_undegraded":"2024-09-16T11:08:53.887995+0000","last_fullsized":"2024-09-16T11:08:53.887995+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:08:06.827173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:08:06.827173+0000","last_clean_scrub_stamp":"2024-09-16T11:08:06.827173+0000","objects_scrubbed":0,"log_size":44,"log_dups_size":0,"ondisk_log_size":44,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:11:35.974778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064218800000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":241,"num_read_kb":2845,"num_write":55,"num_write_kb":2044,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"4","length":"7"}]},{"pgid":"2.5","version":"0'0","reported_seq":35,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T11:08:52.879439+0000","last_change":"2024-09-16T11:07:46.455055+0000","last_active":"2024-09-16T11:08:52.879439+0000","last_peered":"2024-09-16T11:08:52.879439+0000","last_clean":"2024-09-16T11:08:52.879439+0000","last_became_active":"2024-09-16T11:07:44.446498+0000","last_became_peered":"2024-09-16T11:07:44.446498+0000","last_unstale":"2024-09-16T11:08:52.879439+0000","last_undegraded":"2024-09-16T11:08:52.879439+0000","last_fullsized":"2024-09-16T11:08:52.879439+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:07:43.427920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:07:43.427920+0000","last_clean_scrub_stamp":"2024-09-16T11:07:43.427920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:40:08.726582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00084251400000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"32'67","reported_seq":276,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T11:08:53.883465+0000","last_change":"2024-09-16T11:08:53.883643+0000","last_active":"2024-09-16T11:08:53.883465+0000","last_peered":"2024-09-16T11:08:53.883465+0000","last_clean":"2024-09-16T11:08:53.883465+0000","last_became_active":"2024-09-16T11:08:07.853470+0000","last_became_peered":"2024-09-16T11:08:07.853470+0000","last_unstale":"2024-09-16T11:08:53.883465+0000","last_undegraded":"2024-09-16T11:08:53.883465+0000","last_fullsized":"2024-09-16T11:08:53.883465+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:08:06.827173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:08:06.827173+0000","last_clean_scrub_stamp":"2024-09-16T11:08:06.827173+0000","objects_scrubbed":0,"log_size":67,"log_dups_size":0,"ondisk_log_size":67,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:46:06.267425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001033672,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":360,"num_read_kb":1468,"num_write":100,"num_write_kb":888,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"4","length":"7"}]},{"pgid":"2.4","version":"0'0","reported_seq":35,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T11:08:52.879275+0000","last_change":"2024-09-16T11:07:46.454774+0000","last_active":"2024-09-16T11:08:52.879275+0000","last_peered":"2024-09-16T11:08:52.879275+0000","last_clean":"2024-09-16T11:08:52.879275+0000","last_became_active":"2024-09-16T11:07:44.446413+0000","last_became_peered":"2024-09-16T11:07:44.446413+0000","last_unstale":"2024-09-16T11:08:52.879275+0000","last_undegraded":"2024-09-16T11:08:52.879275+0000","last_fullsized":"2024-09-16T11:08:52.879275+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:07:43.427920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:07:43.427920+0000","last_clean_scrub_stamp":"2024-09-16T11:07:43.427920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:53:07.204424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059042800000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"34'78","reported_seq":487,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T11:08:53.887556+0000","last_change":"2024-09-16T11:08:53.887743+0000","last_active":"2024-09-16T11:08:53.887556+0000","last_peered":"2024-09-16T11:08:53.887556+0000","last_clean":"2024-09-16T11:08:53.887556+0000","last_became_active":"2024-09-16T11:08:07.855280+0000","last_became_peered":"2024-09-16T11:08:07.855280+0000","last_unstale":"2024-09-16T11:08:53.887556+0000","last_undegraded":"2024-09-16T11:08:53.887556+0000","last_fullsized":"2024-09-16T11:08:53.887556+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:08:06.827173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:08:06.827173+0000","last_clean_scrub_stamp":"2024-09-16T11:08:06.827173+0000","objects_scrubbed":0,"log_size":78,"log_dups_size":0,"ondisk_log_size":78,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:03:42.315998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00040932799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":396,"num_read_kb":2349,"num_write":79,"num_write_kb":1069,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"4","length":"7"}]},{"pgid":"3.3","version":"35'86","reported_seq":274,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T11:08:54.213465+0000","last_change":"2024-09-16T11:08:53.899598+0000","last_active":"2024-09-16T11:08:54.213465+0000","last_peered":"2024-09-16T11:08:54.213465+0000","last_clean":"2024-09-16T11:08:54.213465+0000","last_became_active":"2024-09-16T11:08:07.847550+0000","last_became_peered":"2024-09-16T11:08:07.847550+0000","last_unstale":"2024-09-16T11:08:54.213465+0000","last_undegraded":"2024-09-16T11:08:54.213465+0000","last_fullsized":"2024-09-16T11:08:54.213465+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:08:06.827173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:08:06.827173+0000","last_clean_scrub_stamp":"2024-09-16T11:08:06.827173+0000","objects_scrubbed":0,"log_size":86,"log_dups_size":0,"ondisk_log_size":86,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000875419,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":164,"num_read_kb":968,"num_write":116,"num_write_kb":2579,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"4","length":"7"}]},{"pgid":"2.2","version":"35'3","reported_seq":48,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T11:08:57.637462+0000","last_change":"2024-09-16T11:08:57.637442+0000","last_active":"2024-09-16T11:08:57.637462+0000","last_peered":"2024-09-16T11:08:57.637462+0000","last_clean":"2024-09-16T11:08:57.637462+0000","last_became_active":"2024-09-16T11:07:44.449069+0000","last_became_peered":"2024-09-16T11:07:44.449069+0000","last_unstale":"2024-09-16T11:08:57.637462+0000","last_undegraded":"2024-09-16T11:08:57.637462+0000","last_fullsized":"2024-09-16T11:08:57.637462+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"35'3","last_scrub_stamp":"2024-09-16T11:08:57.637376+0000","last_deep_scrub":"35'3","last_deep_scrub_stamp":"2024-09-16T11:08:57.637376+0000","last_clean_scrub_stamp":"2024-09-16T11:08:57.637376+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:59:04.639271+0000","scrub_duration":0.0023648480000000001,"objects_trimmed":0,"snaptrim_duration":0.00029305100000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.0","version":"34'47","reported_seq":215,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T11:08:53.882727+0000","last_change":"2024-09-16T11:08:53.882919+0000","last_active":"2024-09-16T11:08:53.882727+0000","last_peered":"2024-09-16T11:08:53.882727+0000","last_clean":"2024-09-16T11:08:53.882727+0000","last_became_active":"2024-09-16T11:08:07.855258+0000","last_became_peered":"2024-09-16T11:08:07.855258+0000","last_unstale":"2024-09-16T11:08:53.882727+0000","last_undegraded":"2024-09-16T11:08:53.882727+0000","last_fullsized":"2024-09-16T11:08:53.882727+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:08:06.827173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:08:06.827173+0000","last_clean_scrub_stamp":"2024-09-16T11:08:06.827173+0000","objects_scrubbed":0,"log_size":47,"log_dups_size":0,"ondisk_log_size":47,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:26:07.701673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00075381500000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":141,"num_read_kb":1673,"num_write":53,"num_write_kb":1263,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"4","length":"7"}]},{"pgid":"2.1","version":"0'0","reported_seq":34,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T11:08:52.874218+0000","last_change":"2024-09-16T11:07:46.720289+0000","last_active":"2024-09-16T11:08:52.874218+0000","last_peered":"2024-09-16T11:08:52.874218+0000","last_clean":"2024-09-16T11:08:52.874218+0000","last_became_active":"2024-09-16T11:07:44.447533+0000","last_became_peered":"2024-09-16T11:07:44.447533+0000","last_unstale":"2024-09-16T11:08:52.874218+0000","last_undegraded":"2024-09-16T11:08:52.874218+0000","last_fullsized":"2024-09-16T11:08:52.874218+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:07:43.427920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:07:43.427920+0000","last_clean_scrub_stamp":"2024-09-16T11:07:43.427920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:13:44.990400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000230378,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"33'30","reported_seq":171,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T11:08:53.898385+0000","last_change":"2024-09-16T11:08:53.898385+0000","last_active":"2024-09-16T11:08:53.898385+0000","last_peered":"2024-09-16T11:08:53.898385+0000","last_clean":"2024-09-16T11:08:53.898385+0000","last_became_active":"2024-09-16T11:08:07.847955+0000","last_became_peered":"2024-09-16T11:08:07.847955+0000","last_unstale":"2024-09-16T11:08:53.898385+0000","last_undegraded":"2024-09-16T11:08:53.898385+0000","last_fullsized":"2024-09-16T11:08:53.898385+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:08:06.827173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:08:06.827173+0000","last_clean_scrub_stamp":"2024-09-16T11:08:06.827173+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00029393900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":87,"num_read_kb":1164,"num_write":31,"num_write_kb":740,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"4","length":"7"}]},{"pgid":"2.0","version":"0'0","reported_seq":34,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T11:08:52.874061+0000","last_change":"2024-09-16T11:07:46.720264+0000","last_active":"2024-09-16T11:08:52.874061+0000","last_peered":"2024-09-16T11:08:52.874061+0000","last_clean":"2024-09-16T11:08:52.874061+0000","last_became_active":"2024-09-16T11:07:44.448153+0000","last_became_peered":"2024-09-16T11:07:44.448153+0000","last_unstale":"2024-09-16T11:08:52.874061+0000","last_undegraded":"2024-09-16T11:08:52.874061+0000","last_fullsized":"2024-09-16T11:08:52.874061+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:07:43.427920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:07:43.427920+0000","last_clean_scrub_stamp":"2024-09-16T11:07:43.427920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T21:09:42.536129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00024792500000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"34'46","reported_seq":281,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T11:08:53.883682+0000","last_change":"2024-09-16T11:08:53.883865+0000","last_active":"2024-09-16T11:08:53.883682+0000","last_peered":"2024-09-16T11:08:53.883682+0000","last_clean":"2024-09-16T11:08:53.883682+0000","last_became_active":"2024-09-16T11:08:07.853307+0000","last_became_peered":"2024-09-16T11:08:07.853307+0000","last_unstale":"2024-09-16T11:08:53.883682+0000","last_undegraded":"2024-09-16T11:08:53.883682+0000","last_fullsized":"2024-09-16T11:08:53.883682+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:08:06.827173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:08:06.827173+0000","last_clean_scrub_stamp":"2024-09-16T11:08:06.827173+0000","objects_scrubbed":0,"log_size":46,"log_dups_size":0,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:33:25.601273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011803549999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":203,"num_read_kb":1989,"num_write":39,"num_write_kb":217,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"4","length":"7"}]},{"pgid":"2.3","version":"13'1","reported_seq":36,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T11:08:52.879069+0000","last_change":"2024-09-16T11:07:46.454516+0000","last_active":"2024-09-16T11:08:52.879069+0000","last_peered":"2024-09-16T11:08:52.879069+0000","last_clean":"2024-09-16T11:08:52.879069+0000","last_became_active":"2024-09-16T11:07:44.446367+0000","last_became_peered":"2024-09-16T11:07:44.446367+0000","last_unstale":"2024-09-16T11:08:52.879069+0000","last_undegraded":"2024-09-16T11:08:52.879069+0000","last_fullsized":"2024-09-16T11:08:52.879069+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:07:43.427920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:07:43.427920+0000","last_clean_scrub_stamp":"2024-09-16T11:07:43.427920+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:06:21.202391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00038771200000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"10'52","reported_seq":100,"reported_epoch":34,"state":"active+clean","last_fresh":"2024-09-16T11:08:52.878764+0000","last_change":"2024-09-16T11:07:39.420884+0000","last_active":"2024-09-16T11:08:52.878764+0000","last_peered":"2024-09-16T11:08:52.878764+0000","last_clean":"2024-09-16T11:08:52.878764+0000","last_became_active":"2024-09-16T11:07:39.420564+0000","last_became_peered":"2024-09-16T11:07:39.420564+0000","last_unstale":"2024-09-16T11:08:52.878764+0000","last_undegraded":"2024-09-16T11:08:52.878764+0000","last_fullsized":"2024-09-16T11:08:52.878764+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:07:38.411971+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:07:38.411971+0000","last_clean_scrub_stamp":"2024-09-16T11:07:38.411971+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:52:50.808244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":56,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"33'41","reported_seq":458,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T11:08:53.888466+0000","last_change":"2024-09-16T11:08:53.888661+0000","last_active":"2024-09-16T11:08:53.888466+0000","last_peered":"2024-09-16T11:08:53.888466+0000","last_clean":"2024-09-16T11:08:53.888466+0000","last_became_active":"2024-09-16T11:08:07.857280+0000","last_became_peered":"2024-09-16T11:08:07.857280+0000","last_unstale":"2024-09-16T11:08:53.888466+0000","last_undegraded":"2024-09-16T11:08:53.888466+0000","last_fullsized":"2024-09-16T11:08:53.888466+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:08:06.827173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:08:06.827173+0000","last_clean_scrub_stamp":"2024-09-16T11:08:06.827173+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:25:26.863744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001178323,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":381,"num_read_kb":3183,"num_write":40,"num_write_kb":936,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"4","length":"7"}]},{"pgid":"3.9","version":"35'18","reported_seq":77,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T11:08:54.187788+0000","last_change":"2024-09-16T11:08:53.898314+0000","last_active":"2024-09-16T11:08:54.187788+0000","last_peered":"2024-09-16T11:08:54.187788+0000","last_clean":"2024-09-16T11:08:54.187788+0000","last_became_active":"2024-09-16T11:08:07.850403+0000","last_became_peered":"2024-09-16T11:08:07.850403+0000","last_unstale":"2024-09-16T11:08:54.187788+0000","last_undegraded":"2024-09-16T11:08:54.187788+0000","last_fullsized":"2024-09-16T11:08:54.187788+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:08:06.827173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:08:06.827173+0000","last_clean_scrub_stamp":"2024-09-16T11:08:06.827173+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025477399999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":55,"num_write":21,"num_write_kb":629,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"4","length":"7"}]},{"pgid":"3.a","version":"34'51","reported_seq":375,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T11:08:53.898790+0000","last_change":"2024-09-16T11:08:53.898790+0000","last_active":"2024-09-16T11:08:53.898790+0000","last_peered":"2024-09-16T11:08:53.898790+0000","last_clean":"2024-09-16T11:08:53.898790+0000","last_became_active":"2024-09-16T11:08:07.857407+0000","last_became_peered":"2024-09-16T11:08:07.857407+0000","last_unstale":"2024-09-16T11:08:53.898790+0000","last_undegraded":"2024-09-16T11:08:53.898790+0000","last_fullsized":"2024-09-16T11:08:53.898790+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:08:06.827173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:08:06.827173+0000","last_clean_scrub_stamp":"2024-09-16T11:08:06.827173+0000","objects_scrubbed":0,"log_size":51,"log_dups_size":0,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055050900000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":289,"num_read_kb":3215,"num_write":55,"num_write_kb":1438,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"4","length":"7"}]},{"pgid":"3.b","version":"34'104","reported_seq":886,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T11:08:53.883730+0000","last_change":"2024-09-16T11:08:53.883898+0000","last_active":"2024-09-16T11:08:53.883730+0000","last_peered":"2024-09-16T11:08:53.883730+0000","last_clean":"2024-09-16T11:08:53.883730+0000","last_became_active":"2024-09-16T11:08:07.855380+0000","last_became_peered":"2024-09-16T11:08:07.855380+0000","last_unstale":"2024-09-16T11:08:53.883730+0000","last_undegraded":"2024-09-16T11:08:53.883730+0000","last_fullsized":"2024-09-16T11:08:53.883730+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:08:06.827173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:08:06.827173+0000","last_clean_scrub_stamp":"2024-09-16T11:08:06.827173+0000","objects_scrubbed":0,"log_size":104,"log_dups_size":0,"ondisk_log_size":104,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:48:03.017466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001457519,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1288,"num_read_kb":4433,"num_write":108,"num_write_kb":1673,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"4","length":"7"}]},{"pgid":"3.c","version":"35'73","reported_seq":412,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T11:08:54.185420+0000","last_change":"2024-09-16T11:08:53.899683+0000","last_active":"2024-09-16T11:08:54.185420+0000","last_peered":"2024-09-16T11:08:54.185420+0000","last_clean":"2024-09-16T11:08:54.185420+0000","last_became_active":"2024-09-16T11:08:07.859197+0000","last_became_peered":"2024-09-16T11:08:07.859197+0000","last_unstale":"2024-09-16T11:08:54.185420+0000","last_undegraded":"2024-09-16T11:08:54.185420+0000","last_fullsized":"2024-09-16T11:08:54.185420+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:08:06.827173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:08:06.827173+0000","last_clean_scrub_stamp":"2024-09-16T11:08:06.827173+0000","objects_scrubbed":0,"log_size":73,"log_dups_size":0,"ondisk_log_size":73,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087272100000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":617,"num_read_kb":1509,"num_write":86,"num_write_kb":959,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"4","length":"7"}]},{"pgid":"3.d","version":"34'78","reported_seq":468,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T11:08:53.883417+0000","last_change":"2024-09-16T11:08:53.883638+0000","last_active":"2024-09-16T11:08:53.883417+0000","last_peered":"2024-09-16T11:08:53.883417+0000","last_clean":"2024-09-16T11:08:53.883417+0000","last_became_active":"2024-09-16T11:08:07.853865+0000","last_became_peered":"2024-09-16T11:08:07.853865+0000","last_unstale":"2024-09-16T11:08:53.883417+0000","last_undegraded":"2024-09-16T11:08:53.883417+0000","last_fullsized":"2024-09-16T11:08:53.883417+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:08:06.827173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:08:06.827173+0000","last_clean_scrub_stamp":"2024-09-16T11:08:06.827173+0000","objects_scrubbed":0,"log_size":78,"log_dups_size":0,"ondisk_log_size":78,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:35:17.633354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001054501,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":785,"num_read_kb":1332,"num_write":70,"num_write_kb":661,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"4","length":"7"}]},{"pgid":"3.e","version":"34'55","reported_seq":325,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T11:08:53.888704+0000","last_change":"2024-09-16T11:08:53.888842+0000","last_active":"2024-09-16T11:08:53.888704+0000","last_peered":"2024-09-16T11:08:53.888704+0000","last_clean":"2024-09-16T11:08:53.888704+0000","last_became_active":"2024-09-16T11:08:07.847486+0000","last_became_peered":"2024-09-16T11:08:07.847486+0000","last_unstale":"2024-09-16T11:08:53.888704+0000","last_undegraded":"2024-09-16T11:08:53.888704+0000","last_fullsized":"2024-09-16T11:08:53.888704+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:08:06.827173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:08:06.827173+0000","last_clean_scrub_stamp":"2024-09-16T11:08:06.827173+0000","objects_scrubbed":0,"log_size":55,"log_dups_size":0,"ondisk_log_size":55,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:40:57.767491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001186241,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":234,"num_read_kb":2061,"num_write":49,"num_write_kb":758,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"4","length":"7"}]},{"pgid":"3.f","version":"29'29","reported_seq":231,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T11:08:53.900154+0000","last_change":"2024-09-16T11:08:53.900154+0000","last_active":"2024-09-16T11:08:53.900154+0000","last_peered":"2024-09-16T11:08:53.900154+0000","last_clean":"2024-09-16T11:08:53.900154+0000","last_became_active":"2024-09-16T11:08:07.859480+0000","last_became_peered":"2024-09-16T11:08:07.859480+0000","last_unstale":"2024-09-16T11:08:53.900154+0000","last_undegraded":"2024-09-16T11:08:53.900154+0000","last_fullsized":"2024-09-16T11:08:53.900154+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:08:06.827173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:08:06.827173+0000","last_clean_scrub_stamp":"2024-09-16T11:08:06.827173+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001035411,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":155,"num_read_kb":866,"num_write":30,"num_write_kb":312,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"4","length":"7"}]},{"pgid":"3.10","version":"33'33","reported_seq":170,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T11:08:53.882281+0000","last_change":"2024-09-16T11:08:53.882549+0000","last_active":"2024-09-16T11:08:53.882281+0000","last_peered":"2024-09-16T11:08:53.882281+0000","last_clean":"2024-09-16T11:08:53.882281+0000","last_became_active":"2024-09-16T11:08:07.855591+0000","last_became_peered":"2024-09-16T11:08:07.855591+0000","last_unstale":"2024-09-16T11:08:53.882281+0000","last_undegraded":"2024-09-16T11:08:53.882281+0000","last_fullsized":"2024-09-16T11:08:53.882281+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:08:06.827173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:08:06.827173+0000","last_clean_scrub_stamp":"2024-09-16T11:08:06.827173+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":0,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:20:16.980280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045890699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":83,"num_read_kb":512,"num_write":38,"num_write_kb":467,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"4","length":"7"}]},{"pgid":"3.11","version":"33'88","reported_seq":957,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T11:08:53.888331+0000","last_change":"2024-09-16T11:08:53.888485+0000","last_active":"2024-09-16T11:08:53.888331+0000","last_peered":"2024-09-16T11:08:53.888331+0000","last_clean":"2024-09-16T11:08:53.888331+0000","last_became_active":"2024-09-16T11:08:07.856626+0000","last_became_peered":"2024-09-16T11:08:07.856626+0000","last_unstale":"2024-09-16T11:08:53.888331+0000","last_undegraded":"2024-09-16T11:08:53.888331+0000","last_fullsized":"2024-09-16T11:08:53.888331+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:08:06.827173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:08:06.827173+0000","last_clean_scrub_stamp":"2024-09-16T11:08:06.827173+0000","objects_scrubbed":0,"log_size":88,"log_dups_size":0,"ondisk_log_size":88,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:27:32.110109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001031704,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1527,"num_read_kb":3994,"num_write":85,"num_write_kb":292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"4","length":"7"}]},{"pgid":"3.12","version":"34'190","reported_seq":378,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T11:08:53.898605+0000","last_change":"2024-09-16T11:08:53.898605+0000","last_active":"2024-09-16T11:08:53.898605+0000","last_peered":"2024-09-16T11:08:53.898605+0000","last_clean":"2024-09-16T11:08:53.898605+0000","last_became_active":"2024-09-16T11:08:07.857698+0000","last_became_peered":"2024-09-16T11:08:07.857698+0000","last_unstale":"2024-09-16T11:08:53.898605+0000","last_undegraded":"2024-09-16T11:08:53.898605+0000","last_fullsized":"2024-09-16T11:08:53.898605+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:08:06.827173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:08:06.827173+0000","last_clean_scrub_stamp":"2024-09-16T11:08:06.827173+0000","objects_scrubbed":0,"log_size":190,"log_dups_size":0,"ondisk_log_size":190,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00040617399999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":912,"num_read_kb":991,"num_write":288,"num_write_kb":407,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"4","length":"7"}]},{"pgid":"3.13","version":"34'232","reported_seq":692,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-16T11:08:53.882779+0000","last_change":"2024-09-16T11:08:53.883037+0000","last_active":"2024-09-16T11:08:53.882779+0000","last_peered":"2024-09-16T11:08:53.882779+0000","last_clean":"2024-09-16T11:08:53.882779+0000","last_became_active":"2024-09-16T11:08:07.853749+0000","last_became_peered":"2024-09-16T11:08:07.853749+0000","last_unstale":"2024-09-16T11:08:53.882779+0000","last_undegraded":"2024-09-16T11:08:53.882779+0000","last_fullsized":"2024-09-16T11:08:53.882779+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:08:06.827173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:08:06.827173+0000","last_clean_scrub_stamp":"2024-09-16T11:08:06.827173+0000","objects_scrubbed":0,"log_size":232,"log_dups_size":0,"ondisk_log_size":232,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:24:10.327154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00063776800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1659,"num_read_kb":2488,"num_write":313,"num_write_kb":1774,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"4","length":"7"}]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":16005,"num_read_kb":49038,"num_write":3041,"num_write_kb":27918,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2483,"ondisk_log_size":2483,"up":64,"acting":64,"num_store_stats":3},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":16,"acting":16,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":56,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":52,"ondisk_log_size":52,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359738388,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301816,"kb_used_data":288,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94070024,"statfs":{"total":96636764160,"available":96327704576,"internally_reserved":0,"allocated":294912,"data_stored":113805,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 11:08:40 2024","interfaces":[{"interface":"back","average":{"1min":0.221,"5min":0.221,"15min":0.221},"min":{"1min":0.14599999999999999,"5min":0.14599999999999999,"15min":0.14599999999999999},"max":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"last":0.28299999999999997},{"interface":"front","average":{"1min":0.23200000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"min":{"1min":0.17000000000000001,"5min":0.17000000000000001,"15min":0.17000000000000001},"max":{"1min":0.31,"5min":0.31,"15min":0.31},"last":0.24199999999999999}]},{"osd":1,"last update":"Mon Sep 16 11:08:40 2024","interfaces":[{"interface":"back","average":{"1min":0.23799999999999999,"5min":0.23799999999999999,"15min":0.23799999999999999},"min":{"1min":0.159,"5min":0.159,"15min":0.159},"max":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"last":0.22800000000000001},{"interface":"front","average":{"1min":0.252,"5min":0.252,"15min":0.252},"min":{"1min":0.184,"5min":0.184,"15min":0.184},"max":{"1min":0.376,"5min":0.376,"15min":0.376},"last":0.253}]}]},{"osd":1,"up_from":8,"seq":34359738389,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":306556,"kb_used_data":876,"kb_used_omap":1,"kb_used_meta":305662,"kb_avail":94065284,"statfs":{"total":96636764160,"available":96322850816,"internally_reserved":0,"allocated":897024,"data_stored":704953,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":312998232},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 11:08:37 2024","interfaces":[{"interface":"back","average":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"min":{"1min":0.17399999999999999,"5min":0.17399999999999999,"15min":0.17399999999999999},"max":{"1min":0.376,"5min":0.376,"15min":0.376},"last":0.23100000000000001},{"interface":"front","average":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"min":{"1min":0.17599999999999999,"5min":0.17599999999999999,"15min":0.17599999999999999},"max":{"1min":0.376,"5min":0.376,"15min":0.376},"last":0.27600000000000002}]},{"osd":2,"last update":"Mon Sep 16 11:08:37 2024","interfaces":[{"interface":"back","average":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"min":{"1min":0.16600000000000001,"5min":0.16600000000000001,"15min":0.16600000000000001},"max":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"last":0.246},{"interface":"front","average":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"min":{"1min":0.191,"5min":0.191,"15min":0.191},"max":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"last":0.26700000000000002}]}]},{"osd":0,"up_from":8,"seq":34359738388,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":306560,"kb_used_data":868,"kb_used_omap":1,"kb_used_meta":305662,"kb_avail":94065280,"statfs":{"total":96636764160,"available":96322846720,"internally_reserved":0,"allocated":888832,"data_stored":704173,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":312998232},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Sep 16 11:08:40 2024","interfaces":[{"interface":"back","average":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"min":{"1min":0.14599999999999999,"5min":0.14599999999999999,"15min":0.14599999999999999},"max":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"last":0.25900000000000001},{"interface":"front","average":{"1min":0.31,"5min":0.31,"15min":0.31},"min":{"1min":0.16600000000000001,"5min":0.16600000000000001,"15min":0.16600000000000001},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.20899999999999999}]},{"osd":2,"last update":"Mon Sep 16 11:08:40 2024","interfaces":[{"interface":"back","average":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"min":{"1min":0.20300000000000001,"5min":0.20300000000000001,"15min":0.20300000000000001},"max":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"last":0.26900000000000002},{"interface":"front","average":{"1min":0.309,"5min":0.309,"15min":0.309},"min":{"1min":0.186,"5min":0.186,"15min":0.186},"max":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"last":0.223}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T11:08:58.382 INFO:tasks.ceph:pgid 3.1f last_scrub_stamp 2024-09-16T11:08:06.827173+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=8, tm_sec=6, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=8, tm_sec=55, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T11:08:58.382 INFO:tasks.ceph:pgid 3.1e last_scrub_stamp 2024-09-16T11:08:06.827173+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=8, tm_sec=6, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=8, tm_sec=55, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T11:08:58.382 INFO:tasks.ceph:pgid 3.1d last_scrub_stamp 2024-09-16T11:08:06.827173+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=8, tm_sec=6, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=8, tm_sec=55, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T11:08:58.382 INFO:tasks.ceph:pgid 3.1c last_scrub_stamp 2024-09-16T11:08:06.827173+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=8, tm_sec=6, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=8, tm_sec=55, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T11:08:58.382 INFO:tasks.ceph:pgid 3.1b last_scrub_stamp 2024-09-16T11:08:06.827173+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=8, tm_sec=6, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=8, tm_sec=55, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T11:08:58.383 INFO:tasks.ceph:pgid 3.1a last_scrub_stamp 2024-09-16T11:08:06.827173+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=8, tm_sec=6, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=8, tm_sec=55, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T11:08:58.383 INFO:tasks.ceph:pgid 3.19 last_scrub_stamp 2024-09-16T11:08:06.827173+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=8, tm_sec=6, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=8, tm_sec=55, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T11:08:58.383 INFO:tasks.ceph:pgid 3.18 last_scrub_stamp 2024-09-16T11:08:06.827173+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=8, tm_sec=6, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=8, tm_sec=55, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T11:08:58.383 INFO:tasks.ceph:pgid 3.17 last_scrub_stamp 2024-09-16T11:08:06.827173+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=8, tm_sec=6, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=8, tm_sec=55, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T11:08:58.383 INFO:tasks.ceph:pgid 3.16 last_scrub_stamp 2024-09-16T11:08:06.827173+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=8, tm_sec=6, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=8, tm_sec=55, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T11:08:58.383 INFO:tasks.ceph:pgid 3.15 last_scrub_stamp 2024-09-16T11:08:06.827173+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=8, tm_sec=6, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=8, tm_sec=55, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T11:08:58.383 INFO:tasks.ceph:pgid 3.14 last_scrub_stamp 2024-09-16T11:08:06.827173+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=8, tm_sec=6, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=8, tm_sec=55, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T11:08:58.383 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2024-09-16T11:07:43.427920+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=7, tm_sec=43, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=8, tm_sec=55, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T11:08:58.383 INFO:tasks.ceph:pgid 3.6 last_scrub_stamp 2024-09-16T11:08:06.827173+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=8, tm_sec=6, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=8, tm_sec=55, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T11:08:58.384 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2024-09-16T11:07:43.427920+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=7, tm_sec=43, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=8, tm_sec=55, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T11:08:58.384 INFO:tasks.ceph:pgid 3.7 last_scrub_stamp 2024-09-16T11:08:06.827173+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=8, tm_sec=6, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=8, tm_sec=55, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T11:08:58.384 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2024-09-16T11:07:43.427920+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=7, tm_sec=43, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=8, tm_sec=55, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T11:08:58.384 INFO:tasks.ceph:pgid 3.4 last_scrub_stamp 2024-09-16T11:08:06.827173+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=8, tm_sec=6, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=8, tm_sec=55, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T11:08:58.384 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2024-09-16T11:07:43.427920+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=7, tm_sec=43, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=8, tm_sec=55, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T11:08:58.384 INFO:tasks.ceph:pgid 3.5 last_scrub_stamp 2024-09-16T11:08:06.827173+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=8, tm_sec=6, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=8, tm_sec=55, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T11:08:58.384 INFO:tasks.ceph:pgid 3.3 last_scrub_stamp 2024-09-16T11:08:06.827173+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=8, tm_sec=6, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=8, tm_sec=55, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T11:08:58.384 INFO:tasks.ceph:pgid 3.0 last_scrub_stamp 2024-09-16T11:08:06.827173+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=8, tm_sec=6, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=8, tm_sec=55, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T11:08:58.385 INFO:tasks.ceph:pgid 2.1 last_scrub_stamp 2024-09-16T11:07:43.427920+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=7, tm_sec=43, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=8, tm_sec=55, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T11:08:58.385 INFO:tasks.ceph:pgid 3.1 last_scrub_stamp 2024-09-16T11:08:06.827173+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=8, tm_sec=6, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=8, tm_sec=55, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T11:08:58.385 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2024-09-16T11:07:43.427920+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=7, tm_sec=43, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=8, tm_sec=55, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T11:08:58.385 INFO:tasks.ceph:pgid 3.2 last_scrub_stamp 2024-09-16T11:08:06.827173+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=8, tm_sec=6, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=8, tm_sec=55, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T11:08:58.385 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2024-09-16T11:07:43.427920+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=7, tm_sec=43, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=8, tm_sec=55, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T11:08:58.385 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-09-16T11:07:38.411971+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=7, tm_sec=38, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=8, tm_sec=55, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T11:08:58.385 INFO:tasks.ceph:pgid 3.8 last_scrub_stamp 2024-09-16T11:08:06.827173+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=8, tm_sec=6, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=8, tm_sec=55, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T11:08:58.385 INFO:tasks.ceph:pgid 3.9 last_scrub_stamp 2024-09-16T11:08:06.827173+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=8, tm_sec=6, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=8, tm_sec=55, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T11:08:58.385 INFO:tasks.ceph:pgid 3.a last_scrub_stamp 2024-09-16T11:08:06.827173+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=8, tm_sec=6, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=8, tm_sec=55, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T11:08:58.385 INFO:tasks.ceph:pgid 3.b last_scrub_stamp 2024-09-16T11:08:06.827173+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=8, tm_sec=6, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=8, tm_sec=55, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T11:08:58.386 INFO:tasks.ceph:pgid 3.c last_scrub_stamp 2024-09-16T11:08:06.827173+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=8, tm_sec=6, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=8, tm_sec=55, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T11:08:58.386 INFO:tasks.ceph:pgid 3.d last_scrub_stamp 2024-09-16T11:08:06.827173+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=8, tm_sec=6, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=8, tm_sec=55, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T11:08:58.386 INFO:tasks.ceph:pgid 3.e last_scrub_stamp 2024-09-16T11:08:06.827173+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=8, tm_sec=6, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=8, tm_sec=55, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T11:08:58.386 INFO:tasks.ceph:pgid 3.f last_scrub_stamp 2024-09-16T11:08:06.827173+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=8, tm_sec=6, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=8, tm_sec=55, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T11:08:58.386 INFO:tasks.ceph:pgid 3.10 last_scrub_stamp 2024-09-16T11:08:06.827173+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=8, tm_sec=6, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=8, tm_sec=55, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T11:08:58.386 INFO:tasks.ceph:pgid 3.11 last_scrub_stamp 2024-09-16T11:08:06.827173+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=8, tm_sec=6, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=8, tm_sec=55, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T11:08:58.386 INFO:tasks.ceph:pgid 3.12 last_scrub_stamp 2024-09-16T11:08:06.827173+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=8, tm_sec=6, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=8, tm_sec=55, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T11:08:58.386 INFO:tasks.ceph:pgid 3.13 last_scrub_stamp 2024-09-16T11:08:06.827173+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=8, tm_sec=6, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=8, tm_sec=55, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T11:08:58.386 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-09-16T11:09:18.388 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T11:09:18.657 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-16T11:09:18.657 INFO:teuthology.orchestra.run.smithi102.stderr:dumped all 2024-09-16T11:09:18.669 INFO:teuthology.orchestra.run.smithi102.stdout:{"pg_ready":true,"pg_map":{"version":85,"stamp":"2024-09-16T11:09:17.664959+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":16061,"num_read_kb":49084,"num_write":3122,"num_write_kb":29026,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2542,"ondisk_log_size":2542,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":914936,"kb_used_data":2080,"kb_used_omap":4,"kb_used_meta":912827,"kb_avail":282200584,"statfs":{"total":289910292480,"available":288973398016,"internally_reserved":0,"allocated":2129920,"data_stored":1544297,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":5112,"internal_metadata":934734856},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002826"},"pg_stats":[{"pgid":"3.1f","version":"35'35","reported_seq":218,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-16T11:09:03.794339+0000","last_change":"2024-09-16T11:08:53.899921+0000","last_active":"2024-09-16T11:09:03.794339+0000","last_peered":"2024-09-16T11:09:03.794339+0000","last_clean":"2024-09-16T11:09:03.794339+0000","last_became_active":"2024-09-16T11:08:07.859102+0000","last_became_peered":"2024-09-16T11:08:07.859102+0000","last_unstale":"2024-09-16T11:09:03.794339+0000","last_undegraded":"2024-09-16T11:09:03.794339+0000","last_fullsized":"2024-09-16T11:09:03.794339+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:08:06.827173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:08:06.827173+0000","last_clean_scrub_stamp":"2024-09-16T11:08:06.827173+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00095524,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":130,"num_read_kb":1007,"num_write":36,"num_write_kb":861,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"35'187","reported_seq":858,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-16T11:09:03.588517+0000","last_change":"2024-09-16T11:08:53.888422+0000","last_active":"2024-09-16T11:09:03.588517+0000","last_peered":"2024-09-16T11:09:03.588517+0000","last_clean":"2024-09-16T11:09:03.588517+0000","last_became_active":"2024-09-16T11:08:07.856092+0000","last_became_peered":"2024-09-16T11:08:07.856092+0000","last_unstale":"2024-09-16T11:09:03.588517+0000","last_undegraded":"2024-09-16T11:09:03.588517+0000","last_fullsized":"2024-09-16T11:09:03.588517+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:08:06.827173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:08:06.827173+0000","last_clean_scrub_stamp":"2024-09-16T11:08:06.827173+0000","objects_scrubbed":0,"log_size":187,"log_dups_size":0,"ondisk_log_size":187,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001038806,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1989,"num_read_kb":2691,"num_write":225,"num_write_kb":434,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1d","version":"35'287","reported_seq":363,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-16T11:09:03.588499+0000","last_change":"2024-09-16T11:08:53.888037+0000","last_active":"2024-09-16T11:09:03.588499+0000","last_peered":"2024-09-16T11:09:03.588499+0000","last_clean":"2024-09-16T11:09:03.588499+0000","last_became_active":"2024-09-16T11:08:07.847513+0000","last_became_peered":"2024-09-16T11:08:07.847513+0000","last_unstale":"2024-09-16T11:09:03.588499+0000","last_undegraded":"2024-09-16T11:09:03.588499+0000","last_fullsized":"2024-09-16T11:09:03.588499+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:08:06.827173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:08:06.827173+0000","last_clean_scrub_stamp":"2024-09-16T11:08:06.827173+0000","objects_scrubbed":0,"log_size":287,"log_dups_size":0,"ondisk_log_size":287,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000667575,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1265,"num_read_kb":1036,"num_write":459,"num_write_kb":602,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"35'36","reported_seq":237,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-16T11:09:02.822249+0000","last_change":"2024-09-16T11:08:53.883362+0000","last_active":"2024-09-16T11:09:02.822249+0000","last_peered":"2024-09-16T11:09:02.822249+0000","last_clean":"2024-09-16T11:09:02.822249+0000","last_became_active":"2024-09-16T11:08:07.855430+0000","last_became_peered":"2024-09-16T11:08:07.855430+0000","last_unstale":"2024-09-16T11:09:02.822249+0000","last_undegraded":"2024-09-16T11:09:02.822249+0000","last_fullsized":"2024-09-16T11:09:02.822249+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:08:06.827173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:08:06.827173+0000","last_clean_scrub_stamp":"2024-09-16T11:08:06.827173+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097256700000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":177,"num_read_kb":260,"num_write":47,"num_write_kb":566,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"34'165","reported_seq":859,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-16T11:09:03.793819+0000","last_change":"2024-09-16T11:08:53.899200+0000","last_active":"2024-09-16T11:09:03.793819+0000","last_peered":"2024-09-16T11:09:03.793819+0000","last_clean":"2024-09-16T11:09:03.793819+0000","last_became_active":"2024-09-16T11:08:07.856827+0000","last_became_peered":"2024-09-16T11:08:07.856827+0000","last_unstale":"2024-09-16T11:09:03.793819+0000","last_undegraded":"2024-09-16T11:09:03.793819+0000","last_fullsized":"2024-09-16T11:09:03.793819+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:08:06.827173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:08:06.827173+0000","last_clean_scrub_stamp":"2024-09-16T11:08:06.827173+0000","objects_scrubbed":0,"log_size":165,"log_dups_size":0,"ondisk_log_size":165,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081398199999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1407,"num_read_kb":2244,"num_write":140,"num_write_kb":887,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"28'13","reported_seq":399,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-16T11:09:02.822209+0000","last_change":"2024-09-16T11:08:53.883198+0000","last_active":"2024-09-16T11:09:02.822209+0000","last_peered":"2024-09-16T11:09:02.822209+0000","last_clean":"2024-09-16T11:09:02.822209+0000","last_became_active":"2024-09-16T11:08:07.853372+0000","last_became_peered":"2024-09-16T11:08:07.853372+0000","last_unstale":"2024-09-16T11:09:02.822209+0000","last_undegraded":"2024-09-16T11:09:02.822209+0000","last_fullsized":"2024-09-16T11:09:02.822209+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:08:06.827173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:08:06.827173+0000","last_clean_scrub_stamp":"2024-09-16T11:08:06.827173+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00082456000000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":325,"num_read_kb":343,"num_write":13,"num_write_kb":81,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.19","version":"35'47","reported_seq":302,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-16T11:09:02.822105+0000","last_change":"2024-09-16T11:08:53.882678+0000","last_active":"2024-09-16T11:09:02.822105+0000","last_peered":"2024-09-16T11:09:02.822105+0000","last_clean":"2024-09-16T11:09:02.822105+0000","last_became_active":"2024-09-16T11:08:07.852874+0000","last_became_peered":"2024-09-16T11:08:07.852874+0000","last_unstale":"2024-09-16T11:09:02.822105+0000","last_undegraded":"2024-09-16T11:09:02.822105+0000","last_fullsized":"2024-09-16T11:09:02.822105+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:08:06.827173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:08:06.827173+0000","last_clean_scrub_stamp":"2024-09-16T11:08:06.827173+0000","objects_scrubbed":0,"log_size":47,"log_dups_size":0,"ondisk_log_size":47,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00052581399999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":214,"num_read_kb":667,"num_write":46,"num_write_kb":746,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"35'18","reported_seq":82,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-16T11:09:03.588435+0000","last_change":"2024-09-16T11:08:53.887745+0000","last_active":"2024-09-16T11:09:03.588435+0000","last_peered":"2024-09-16T11:09:03.588435+0000","last_clean":"2024-09-16T11:09:03.588435+0000","last_became_active":"2024-09-16T11:08:07.848434+0000","last_became_peered":"2024-09-16T11:08:07.848434+0000","last_unstale":"2024-09-16T11:09:03.588435+0000","last_undegraded":"2024-09-16T11:09:03.588435+0000","last_fullsized":"2024-09-16T11:09:03.588435+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:08:06.827173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:08:06.827173+0000","last_clean_scrub_stamp":"2024-09-16T11:08:06.827173+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043492900000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":18,"num_write_kb":60,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.17","version":"31'25","reported_seq":114,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-16T11:09:03.794553+0000","last_change":"2024-09-16T11:08:53.900234+0000","last_active":"2024-09-16T11:09:03.794553+0000","last_peered":"2024-09-16T11:09:03.794553+0000","last_clean":"2024-09-16T11:09:03.794553+0000","last_became_active":"2024-09-16T11:08:07.859097+0000","last_became_peered":"2024-09-16T11:08:07.859097+0000","last_unstale":"2024-09-16T11:09:03.794553+0000","last_undegraded":"2024-09-16T11:09:03.794553+0000","last_fullsized":"2024-09-16T11:09:03.794553+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:08:06.827173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:08:06.827173+0000","last_clean_scrub_stamp":"2024-09-16T11:08:06.827173+0000","objects_scrubbed":0,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001161507,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":30,"num_read_kb":66,"num_write":24,"num_write_kb":325,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"32'25","reported_seq":143,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-16T11:09:03.589023+0000","last_change":"2024-09-16T11:08:53.888681+0000","last_active":"2024-09-16T11:09:03.589023+0000","last_peered":"2024-09-16T11:09:03.589023+0000","last_clean":"2024-09-16T11:09:03.589023+0000","last_became_active":"2024-09-16T11:08:07.856007+0000","last_became_peered":"2024-09-16T11:08:07.856007+0000","last_unstale":"2024-09-16T11:09:03.589023+0000","last_undegraded":"2024-09-16T11:09:03.589023+0000","last_fullsized":"2024-09-16T11:09:03.589023+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:08:06.827173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:08:06.827173+0000","last_clean_scrub_stamp":"2024-09-16T11:08:06.827173+0000","objects_scrubbed":0,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011111560000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":66,"num_read_kb":1713,"num_write":38,"num_write_kb":1539,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"33'31","reported_seq":175,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-16T11:09:03.794000+0000","last_change":"2024-09-16T11:08:53.899692+0000","last_active":"2024-09-16T11:09:03.794000+0000","last_peered":"2024-09-16T11:09:03.794000+0000","last_clean":"2024-09-16T11:09:03.794000+0000","last_became_active":"2024-09-16T11:08:07.858375+0000","last_became_peered":"2024-09-16T11:08:07.858375+0000","last_unstale":"2024-09-16T11:09:03.794000+0000","last_undegraded":"2024-09-16T11:09:03.794000+0000","last_fullsized":"2024-09-16T11:09:03.794000+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:08:06.827173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:08:06.827173+0000","last_clean_scrub_stamp":"2024-09-16T11:08:06.827173+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00079626899999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":92,"num_read_kb":770,"num_write":36,"num_write_kb":1093,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.14","version":"35'141","reported_seq":308,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-16T11:09:02.822320+0000","last_change":"2024-09-16T11:08:53.883413+0000","last_active":"2024-09-16T11:09:02.822320+0000","last_peered":"2024-09-16T11:09:02.822320+0000","last_clean":"2024-09-16T11:09:02.822320+0000","last_became_active":"2024-09-16T11:08:07.853189+0000","last_became_peered":"2024-09-16T11:08:07.853189+0000","last_unstale":"2024-09-16T11:09:02.822320+0000","last_undegraded":"2024-09-16T11:09:02.822320+0000","last_fullsized":"2024-09-16T11:09:02.822320+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:08:06.827173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:08:06.827173+0000","last_clean_scrub_stamp":"2024-09-16T11:08:06.827173+0000","objects_scrubbed":0,"log_size":141,"log_dups_size":0,"ondisk_log_size":141,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087565099999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":523,"num_read_kb":930,"num_write":189,"num_write_kb":943,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":48,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-16T11:09:03.797856+0000","last_change":"2024-09-16T11:09:03.797818+0000","last_active":"2024-09-16T11:09:03.797856+0000","last_peered":"2024-09-16T11:09:03.797856+0000","last_clean":"2024-09-16T11:09:03.797856+0000","last_became_active":"2024-09-16T11:07:44.446646+0000","last_became_peered":"2024-09-16T11:07:44.446646+0000","last_unstale":"2024-09-16T11:09:03.797856+0000","last_undegraded":"2024-09-16T11:09:03.797856+0000","last_fullsized":"2024-09-16T11:09:03.797856+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:09:03.797714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:09:03.797714+0000","last_clean_scrub_stamp":"2024-09-16T11:09:03.797714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:29:49.984164+0000","scrub_duration":0.0062555400000000004,"objects_trimmed":0,"snaptrim_duration":0.000337344,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.6","version":"35'83","reported_seq":153,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-16T11:09:03.794344+0000","last_change":"2024-09-16T11:08:53.899856+0000","last_active":"2024-09-16T11:09:03.794344+0000","last_peered":"2024-09-16T11:09:03.794344+0000","last_clean":"2024-09-16T11:09:03.794344+0000","last_became_active":"2024-09-16T11:08:07.858208+0000","last_became_peered":"2024-09-16T11:08:07.858208+0000","last_unstale":"2024-09-16T11:09:03.794344+0000","last_undegraded":"2024-09-16T11:09:03.794344+0000","last_fullsized":"2024-09-16T11:09:03.794344+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:08:06.827173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:08:06.827173+0000","last_clean_scrub_stamp":"2024-09-16T11:08:06.827173+0000","objects_scrubbed":0,"log_size":83,"log_dups_size":0,"ondisk_log_size":83,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098881699999999991,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":259,"num_read_kb":213,"num_write":114,"num_write_kb":675,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":48,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-16T11:09:07.770754+0000","last_change":"2024-09-16T11:09:07.770730+0000","last_active":"2024-09-16T11:09:07.770754+0000","last_peered":"2024-09-16T11:09:07.770754+0000","last_clean":"2024-09-16T11:09:07.770754+0000","last_became_active":"2024-09-16T11:07:44.446406+0000","last_became_peered":"2024-09-16T11:07:44.446406+0000","last_unstale":"2024-09-16T11:09:07.770754+0000","last_undegraded":"2024-09-16T11:09:07.770754+0000","last_fullsized":"2024-09-16T11:09:07.770754+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:09:07.770642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:09:07.770642+0000","last_clean_scrub_stamp":"2024-09-16T11:09:07.770642+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:29:11.031172+0000","scrub_duration":0.001319305,"objects_trimmed":0,"snaptrim_duration":0.00062301399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"35'44","reported_seq":332,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-16T11:09:03.588534+0000","last_change":"2024-09-16T11:08:53.887995+0000","last_active":"2024-09-16T11:09:03.588534+0000","last_peered":"2024-09-16T11:09:03.588534+0000","last_clean":"2024-09-16T11:09:03.588534+0000","last_became_active":"2024-09-16T11:08:07.856393+0000","last_became_peered":"2024-09-16T11:08:07.856393+0000","last_unstale":"2024-09-16T11:09:03.588534+0000","last_undegraded":"2024-09-16T11:09:03.588534+0000","last_fullsized":"2024-09-16T11:09:03.588534+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:08:06.827173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:08:06.827173+0000","last_clean_scrub_stamp":"2024-09-16T11:08:06.827173+0000","objects_scrubbed":0,"log_size":44,"log_dups_size":0,"ondisk_log_size":44,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064218800000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":241,"num_read_kb":2845,"num_write":55,"num_write_kb":2044,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":38,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-16T11:08:57.821189+0000","last_change":"2024-09-16T11:07:46.455055+0000","last_active":"2024-09-16T11:08:57.821189+0000","last_peered":"2024-09-16T11:08:57.821189+0000","last_clean":"2024-09-16T11:08:57.821189+0000","last_became_active":"2024-09-16T11:07:44.446498+0000","last_became_peered":"2024-09-16T11:07:44.446498+0000","last_unstale":"2024-09-16T11:08:57.821189+0000","last_undegraded":"2024-09-16T11:08:57.821189+0000","last_fullsized":"2024-09-16T11:08:57.821189+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:07:43.427920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:07:43.427920+0000","last_clean_scrub_stamp":"2024-09-16T11:07:43.427920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00084251400000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"32'67","reported_seq":281,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-16T11:09:02.822511+0000","last_change":"2024-09-16T11:08:53.883643+0000","last_active":"2024-09-16T11:09:02.822511+0000","last_peered":"2024-09-16T11:09:02.822511+0000","last_clean":"2024-09-16T11:09:02.822511+0000","last_became_active":"2024-09-16T11:08:07.853470+0000","last_became_peered":"2024-09-16T11:08:07.853470+0000","last_unstale":"2024-09-16T11:09:02.822511+0000","last_undegraded":"2024-09-16T11:09:02.822511+0000","last_fullsized":"2024-09-16T11:09:02.822511+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:08:06.827173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:08:06.827173+0000","last_clean_scrub_stamp":"2024-09-16T11:08:06.827173+0000","objects_scrubbed":0,"log_size":67,"log_dups_size":0,"ondisk_log_size":67,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001033672,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":360,"num_read_kb":1468,"num_write":100,"num_write_kb":888,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":38,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-16T11:08:57.821511+0000","last_change":"2024-09-16T11:07:46.454774+0000","last_active":"2024-09-16T11:08:57.821511+0000","last_peered":"2024-09-16T11:08:57.821511+0000","last_clean":"2024-09-16T11:08:57.821511+0000","last_became_active":"2024-09-16T11:07:44.446413+0000","last_became_peered":"2024-09-16T11:07:44.446413+0000","last_unstale":"2024-09-16T11:08:57.821511+0000","last_undegraded":"2024-09-16T11:08:57.821511+0000","last_fullsized":"2024-09-16T11:08:57.821511+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:07:43.427920+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:07:43.427920+0000","last_clean_scrub_stamp":"2024-09-16T11:07:43.427920+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059042800000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"34'78","reported_seq":492,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-16T11:09:03.588593+0000","last_change":"2024-09-16T11:08:53.887743+0000","last_active":"2024-09-16T11:09:03.588593+0000","last_peered":"2024-09-16T11:09:03.588593+0000","last_clean":"2024-09-16T11:09:03.588593+0000","last_became_active":"2024-09-16T11:08:07.855280+0000","last_became_peered":"2024-09-16T11:08:07.855280+0000","last_unstale":"2024-09-16T11:09:03.588593+0000","last_undegraded":"2024-09-16T11:09:03.588593+0000","last_fullsized":"2024-09-16T11:09:03.588593+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:08:06.827173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:08:06.827173+0000","last_clean_scrub_stamp":"2024-09-16T11:08:06.827173+0000","objects_scrubbed":0,"log_size":78,"log_dups_size":0,"ondisk_log_size":78,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00040932799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":396,"num_read_kb":2349,"num_write":79,"num_write_kb":1069,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"35'86","reported_seq":288,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-16T11:09:07.764657+0000","last_change":"2024-09-16T11:09:07.764626+0000","last_active":"2024-09-16T11:09:07.764657+0000","last_peered":"2024-09-16T11:09:07.764657+0000","last_clean":"2024-09-16T11:09:07.764657+0000","last_became_active":"2024-09-16T11:08:07.847550+0000","last_became_peered":"2024-09-16T11:08:07.847550+0000","last_unstale":"2024-09-16T11:09:07.764657+0000","last_undegraded":"2024-09-16T11:09:07.764657+0000","last_fullsized":"2024-09-16T11:09:07.764657+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"35'86","last_scrub_stamp":"2024-09-16T11:09:07.764522+0000","last_deep_scrub":"35'86","last_deep_scrub_stamp":"2024-09-16T11:09:07.764522+0000","last_clean_scrub_stamp":"2024-09-16T11:09:07.764522+0000","objects_scrubbed":1,"log_size":86,"log_dups_size":0,"ondisk_log_size":86,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:29:53.950973+0000","scrub_duration":0.0017137980000000001,"objects_trimmed":0,"snaptrim_duration":0.000875419,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":164,"num_read_kb":968,"num_write":116,"num_write_kb":2579,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"35'3","reported_seq":50,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-16T11:08:58.393319+0000","last_change":"2024-09-16T11:08:57.637442+0000","last_active":"2024-09-16T11:08:58.393319+0000","last_peered":"2024-09-16T11:08:58.393319+0000","last_clean":"2024-09-16T11:08:58.393319+0000","last_became_active":"2024-09-16T11:07:44.449069+0000","last_became_peered":"2024-09-16T11:07:44.449069+0000","last_unstale":"2024-09-16T11:08:58.393319+0000","last_undegraded":"2024-09-16T11:08:58.393319+0000","last_fullsized":"2024-09-16T11:08:58.393319+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"35'3","last_scrub_stamp":"2024-09-16T11:08:57.637376+0000","last_deep_scrub":"35'3","last_deep_scrub_stamp":"2024-09-16T11:08:57.637376+0000","last_clean_scrub_stamp":"2024-09-16T11:08:57.637376+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:59:04.639271+0000","scrub_duration":0.0023648480000000001,"objects_trimmed":0,"snaptrim_duration":0.00029305100000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.0","version":"34'47","reported_seq":220,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-16T11:09:02.822147+0000","last_change":"2024-09-16T11:08:53.882919+0000","last_active":"2024-09-16T11:09:02.822147+0000","last_peered":"2024-09-16T11:09:02.822147+0000","last_clean":"2024-09-16T11:09:02.822147+0000","last_became_active":"2024-09-16T11:08:07.855258+0000","last_became_peered":"2024-09-16T11:08:07.855258+0000","last_unstale":"2024-09-16T11:09:02.822147+0000","last_undegraded":"2024-09-16T11:09:02.822147+0000","last_fullsized":"2024-09-16T11:09:02.822147+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:08:06.827173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:08:06.827173+0000","last_clean_scrub_stamp":"2024-09-16T11:08:06.827173+0000","objects_scrubbed":0,"log_size":47,"log_dups_size":0,"ondisk_log_size":47,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00075381500000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":141,"num_read_kb":1673,"num_write":53,"num_write_kb":1263,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":47,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-16T11:09:06.562694+0000","last_change":"2024-09-16T11:09:06.562665+0000","last_active":"2024-09-16T11:09:06.562694+0000","last_peered":"2024-09-16T11:09:06.562694+0000","last_clean":"2024-09-16T11:09:06.562694+0000","last_became_active":"2024-09-16T11:07:44.447533+0000","last_became_peered":"2024-09-16T11:07:44.447533+0000","last_unstale":"2024-09-16T11:09:06.562694+0000","last_undegraded":"2024-09-16T11:09:06.562694+0000","last_fullsized":"2024-09-16T11:09:06.562694+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:09:06.562572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:09:06.562572+0000","last_clean_scrub_stamp":"2024-09-16T11:09:06.562572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:43:14.347543+0000","scrub_duration":0.0013831399999999999,"objects_trimmed":0,"snaptrim_duration":0.000230378,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"33'30","reported_seq":185,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-16T11:09:03.804035+0000","last_change":"2024-09-16T11:09:03.804009+0000","last_active":"2024-09-16T11:09:03.804035+0000","last_peered":"2024-09-16T11:09:03.804035+0000","last_clean":"2024-09-16T11:09:03.804035+0000","last_became_active":"2024-09-16T11:08:07.847955+0000","last_became_peered":"2024-09-16T11:08:07.847955+0000","last_unstale":"2024-09-16T11:09:03.804035+0000","last_undegraded":"2024-09-16T11:09:03.804035+0000","last_fullsized":"2024-09-16T11:09:03.804035+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'30","last_scrub_stamp":"2024-09-16T11:09:03.803902+0000","last_deep_scrub":"33'30","last_deep_scrub_stamp":"2024-09-16T11:09:03.803902+0000","last_clean_scrub_stamp":"2024-09-16T11:09:03.803902+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:43:11.588873+0000","scrub_duration":0.001499385,"objects_trimmed":0,"snaptrim_duration":0.00029393900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":87,"num_read_kb":1164,"num_write":31,"num_write_kb":740,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":47,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-16T11:09:01.573333+0000","last_change":"2024-09-16T11:09:01.573294+0000","last_active":"2024-09-16T11:09:01.573333+0000","last_peered":"2024-09-16T11:09:01.573333+0000","last_clean":"2024-09-16T11:09:01.573333+0000","last_became_active":"2024-09-16T11:07:44.448153+0000","last_became_peered":"2024-09-16T11:07:44.448153+0000","last_unstale":"2024-09-16T11:09:01.573333+0000","last_undegraded":"2024-09-16T11:09:01.573333+0000","last_fullsized":"2024-09-16T11:09:01.573333+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:09:01.573204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:09:01.573204+0000","last_clean_scrub_stamp":"2024-09-16T11:09:01.573204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:59:08.575099+0000","scrub_duration":0.001347532,"objects_trimmed":0,"snaptrim_duration":0.00024792500000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"34'46","reported_seq":296,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-16T11:09:02.822765+0000","last_change":"2024-09-16T11:08:59.685510+0000","last_active":"2024-09-16T11:09:02.822765+0000","last_peered":"2024-09-16T11:09:02.822765+0000","last_clean":"2024-09-16T11:09:02.822765+0000","last_became_active":"2024-09-16T11:08:07.853307+0000","last_became_peered":"2024-09-16T11:08:07.853307+0000","last_unstale":"2024-09-16T11:09:02.822765+0000","last_undegraded":"2024-09-16T11:09:02.822765+0000","last_fullsized":"2024-09-16T11:09:02.822765+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'46","last_scrub_stamp":"2024-09-16T11:08:59.685380+0000","last_deep_scrub":"34'46","last_deep_scrub_stamp":"2024-09-16T11:08:59.685380+0000","last_clean_scrub_stamp":"2024-09-16T11:08:59.685380+0000","objects_scrubbed":0,"log_size":46,"log_dups_size":0,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:43:07.470351+0000","scrub_duration":0.0015725050000000001,"objects_trimmed":0,"snaptrim_duration":0.0011803549999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":203,"num_read_kb":1989,"num_write":39,"num_write_kb":217,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"37'2","reported_seq":50,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-16T11:09:11.724730+0000","last_change":"2024-09-16T11:09:11.724697+0000","last_active":"2024-09-16T11:09:11.724730+0000","last_peered":"2024-09-16T11:09:11.724730+0000","last_clean":"2024-09-16T11:09:11.724730+0000","last_became_active":"2024-09-16T11:07:44.446367+0000","last_became_peered":"2024-09-16T11:07:44.446367+0000","last_unstale":"2024-09-16T11:09:11.724730+0000","last_undegraded":"2024-09-16T11:09:11.724730+0000","last_fullsized":"2024-09-16T11:09:11.724730+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2","last_scrub_stamp":"2024-09-16T11:09:11.724600+0000","last_deep_scrub":"37'2","last_deep_scrub_stamp":"2024-09-16T11:09:11.724600+0000","last_clean_scrub_stamp":"2024-09-16T11:09:11.724600+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:07:08.042481+0000","scrub_duration":0.0026358029999999999,"objects_trimmed":0,"snaptrim_duration":0.00038771200000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"36'54","reported_seq":115,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-16T11:09:02.760758+0000","last_change":"2024-09-16T11:09:02.760678+0000","last_active":"2024-09-16T11:09:02.760758+0000","last_peered":"2024-09-16T11:09:02.760758+0000","last_clean":"2024-09-16T11:09:02.760758+0000","last_became_active":"2024-09-16T11:07:39.420564+0000","last_became_peered":"2024-09-16T11:07:39.420564+0000","last_unstale":"2024-09-16T11:09:02.760758+0000","last_undegraded":"2024-09-16T11:09:02.760758+0000","last_fullsized":"2024-09-16T11:09:02.760758+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'54","last_scrub_stamp":"2024-09-16T11:09:02.760577+0000","last_deep_scrub":"36'54","last_deep_scrub_stamp":"2024-09-16T11:09:02.760577+0000","last_clean_scrub_stamp":"2024-09-16T11:09:02.760577+0000","objects_scrubbed":2,"log_size":54,"log_dups_size":0,"ondisk_log_size":54,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:11:57.859673+0000","scrub_duration":0.0048002319999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":56,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"33'41","reported_seq":463,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-16T11:09:03.588471+0000","last_change":"2024-09-16T11:08:53.888661+0000","last_active":"2024-09-16T11:09:03.588471+0000","last_peered":"2024-09-16T11:09:03.588471+0000","last_clean":"2024-09-16T11:09:03.588471+0000","last_became_active":"2024-09-16T11:08:07.857280+0000","last_became_peered":"2024-09-16T11:08:07.857280+0000","last_unstale":"2024-09-16T11:09:03.588471+0000","last_undegraded":"2024-09-16T11:09:03.588471+0000","last_fullsized":"2024-09-16T11:09:03.588471+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:08:06.827173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:08:06.827173+0000","last_clean_scrub_stamp":"2024-09-16T11:08:06.827173+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001178323,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":381,"num_read_kb":3183,"num_write":40,"num_write_kb":936,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.9","version":"35'18","reported_seq":81,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-16T11:09:03.793436+0000","last_change":"2024-09-16T11:08:53.898314+0000","last_active":"2024-09-16T11:09:03.793436+0000","last_peered":"2024-09-16T11:09:03.793436+0000","last_clean":"2024-09-16T11:09:03.793436+0000","last_became_active":"2024-09-16T11:08:07.850403+0000","last_became_peered":"2024-09-16T11:08:07.850403+0000","last_unstale":"2024-09-16T11:09:03.793436+0000","last_undegraded":"2024-09-16T11:09:03.793436+0000","last_fullsized":"2024-09-16T11:09:03.793436+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:08:06.827173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:08:06.827173+0000","last_clean_scrub_stamp":"2024-09-16T11:08:06.827173+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025477399999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":55,"num_write":21,"num_write_kb":629,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.a","version":"34'51","reported_seq":379,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-16T11:09:03.793716+0000","last_change":"2024-09-16T11:08:53.898790+0000","last_active":"2024-09-16T11:09:03.793716+0000","last_peered":"2024-09-16T11:09:03.793716+0000","last_clean":"2024-09-16T11:09:03.793716+0000","last_became_active":"2024-09-16T11:08:07.857407+0000","last_became_peered":"2024-09-16T11:08:07.857407+0000","last_unstale":"2024-09-16T11:09:03.793716+0000","last_undegraded":"2024-09-16T11:09:03.793716+0000","last_fullsized":"2024-09-16T11:09:03.793716+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:08:06.827173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:08:06.827173+0000","last_clean_scrub_stamp":"2024-09-16T11:08:06.827173+0000","objects_scrubbed":0,"log_size":51,"log_dups_size":0,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055050900000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":289,"num_read_kb":3215,"num_write":55,"num_write_kb":1438,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"34'104","reported_seq":891,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-16T11:09:02.822592+0000","last_change":"2024-09-16T11:08:53.883898+0000","last_active":"2024-09-16T11:09:02.822592+0000","last_peered":"2024-09-16T11:09:02.822592+0000","last_clean":"2024-09-16T11:09:02.822592+0000","last_became_active":"2024-09-16T11:08:07.855380+0000","last_became_peered":"2024-09-16T11:08:07.855380+0000","last_unstale":"2024-09-16T11:09:02.822592+0000","last_undegraded":"2024-09-16T11:09:02.822592+0000","last_fullsized":"2024-09-16T11:09:02.822592+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:08:06.827173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:08:06.827173+0000","last_clean_scrub_stamp":"2024-09-16T11:08:06.827173+0000","objects_scrubbed":0,"log_size":104,"log_dups_size":0,"ondisk_log_size":104,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001457519,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1288,"num_read_kb":4433,"num_write":108,"num_write_kb":1673,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"35'73","reported_seq":426,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-16T11:09:11.792421+0000","last_change":"2024-09-16T11:09:11.792370+0000","last_active":"2024-09-16T11:09:11.792421+0000","last_peered":"2024-09-16T11:09:11.792421+0000","last_clean":"2024-09-16T11:09:11.792421+0000","last_became_active":"2024-09-16T11:08:07.859197+0000","last_became_peered":"2024-09-16T11:08:07.859197+0000","last_unstale":"2024-09-16T11:09:11.792421+0000","last_undegraded":"2024-09-16T11:09:11.792421+0000","last_fullsized":"2024-09-16T11:09:11.792421+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"35'73","last_scrub_stamp":"2024-09-16T11:09:11.792260+0000","last_deep_scrub":"35'73","last_deep_scrub_stamp":"2024-09-16T11:09:11.792260+0000","last_clean_scrub_stamp":"2024-09-16T11:09:11.792260+0000","objects_scrubbed":0,"log_size":73,"log_dups_size":0,"ondisk_log_size":73,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:29:15.052789+0000","scrub_duration":0.001460004,"objects_trimmed":0,"snaptrim_duration":0.00087272100000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":617,"num_read_kb":1509,"num_write":86,"num_write_kb":959,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.d","version":"34'78","reported_seq":473,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-16T11:09:02.822564+0000","last_change":"2024-09-16T11:08:53.883638+0000","last_active":"2024-09-16T11:09:02.822564+0000","last_peered":"2024-09-16T11:09:02.822564+0000","last_clean":"2024-09-16T11:09:02.822564+0000","last_became_active":"2024-09-16T11:08:07.853865+0000","last_became_peered":"2024-09-16T11:08:07.853865+0000","last_unstale":"2024-09-16T11:09:02.822564+0000","last_undegraded":"2024-09-16T11:09:02.822564+0000","last_fullsized":"2024-09-16T11:09:02.822564+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:08:06.827173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:08:06.827173+0000","last_clean_scrub_stamp":"2024-09-16T11:08:06.827173+0000","objects_scrubbed":0,"log_size":78,"log_dups_size":0,"ondisk_log_size":78,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001054501,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":785,"num_read_kb":1332,"num_write":70,"num_write_kb":661,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.e","version":"34'55","reported_seq":340,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-16T11:09:10.614689+0000","last_change":"2024-09-16T11:09:10.614661+0000","last_active":"2024-09-16T11:09:10.614689+0000","last_peered":"2024-09-16T11:09:10.614689+0000","last_clean":"2024-09-16T11:09:10.614689+0000","last_became_active":"2024-09-16T11:08:07.847486+0000","last_became_peered":"2024-09-16T11:08:07.847486+0000","last_unstale":"2024-09-16T11:09:10.614689+0000","last_undegraded":"2024-09-16T11:09:10.614689+0000","last_fullsized":"2024-09-16T11:09:10.614689+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'55","last_scrub_stamp":"2024-09-16T11:09:10.614573+0000","last_deep_scrub":"34'55","last_deep_scrub_stamp":"2024-09-16T11:09:10.614573+0000","last_clean_scrub_stamp":"2024-09-16T11:09:10.614573+0000","objects_scrubbed":0,"log_size":55,"log_dups_size":0,"ondisk_log_size":55,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:29:56.801023+0000","scrub_duration":0.001406416,"objects_trimmed":0,"snaptrim_duration":0.001186241,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":234,"num_read_kb":2061,"num_write":49,"num_write_kb":758,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"29'29","reported_seq":235,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-16T11:09:03.794699+0000","last_change":"2024-09-16T11:08:53.900154+0000","last_active":"2024-09-16T11:09:03.794699+0000","last_peered":"2024-09-16T11:09:03.794699+0000","last_clean":"2024-09-16T11:09:03.794699+0000","last_became_active":"2024-09-16T11:08:07.859480+0000","last_became_peered":"2024-09-16T11:08:07.859480+0000","last_unstale":"2024-09-16T11:09:03.794699+0000","last_undegraded":"2024-09-16T11:09:03.794699+0000","last_fullsized":"2024-09-16T11:09:03.794699+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:08:06.827173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:08:06.827173+0000","last_clean_scrub_stamp":"2024-09-16T11:08:06.827173+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001035411,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":155,"num_read_kb":866,"num_write":30,"num_write_kb":312,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"33'33","reported_seq":175,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-16T11:09:02.822010+0000","last_change":"2024-09-16T11:08:53.882549+0000","last_active":"2024-09-16T11:09:02.822010+0000","last_peered":"2024-09-16T11:09:02.822010+0000","last_clean":"2024-09-16T11:09:02.822010+0000","last_became_active":"2024-09-16T11:08:07.855591+0000","last_became_peered":"2024-09-16T11:08:07.855591+0000","last_unstale":"2024-09-16T11:09:02.822010+0000","last_undegraded":"2024-09-16T11:09:02.822010+0000","last_fullsized":"2024-09-16T11:09:02.822010+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:08:06.827173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:08:06.827173+0000","last_clean_scrub_stamp":"2024-09-16T11:08:06.827173+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":0,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045890699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":83,"num_read_kb":512,"num_write":38,"num_write_kb":467,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.11","version":"33'88","reported_seq":972,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-16T11:09:09.612356+0000","last_change":"2024-09-16T11:09:09.612308+0000","last_active":"2024-09-16T11:09:09.612356+0000","last_peered":"2024-09-16T11:09:09.612356+0000","last_clean":"2024-09-16T11:09:09.612356+0000","last_became_active":"2024-09-16T11:08:07.856626+0000","last_became_peered":"2024-09-16T11:08:07.856626+0000","last_unstale":"2024-09-16T11:09:09.612356+0000","last_undegraded":"2024-09-16T11:09:09.612356+0000","last_fullsized":"2024-09-16T11:09:09.612356+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'88","last_scrub_stamp":"2024-09-16T11:09:09.612205+0000","last_deep_scrub":"33'88","last_deep_scrub_stamp":"2024-09-16T11:09:09.612205+0000","last_clean_scrub_stamp":"2024-09-16T11:09:09.612205+0000","objects_scrubbed":0,"log_size":88,"log_dups_size":0,"ondisk_log_size":88,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:12:04.711301+0000","scrub_duration":0.0014779610000000001,"objects_trimmed":0,"snaptrim_duration":0.001031704,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1527,"num_read_kb":3994,"num_write":85,"num_write_kb":292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"34'190","reported_seq":382,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-16T11:09:03.793526+0000","last_change":"2024-09-16T11:08:53.898605+0000","last_active":"2024-09-16T11:09:03.793526+0000","last_peered":"2024-09-16T11:09:03.793526+0000","last_clean":"2024-09-16T11:09:03.793526+0000","last_became_active":"2024-09-16T11:08:07.857698+0000","last_became_peered":"2024-09-16T11:08:07.857698+0000","last_unstale":"2024-09-16T11:09:03.793526+0000","last_undegraded":"2024-09-16T11:09:03.793526+0000","last_fullsized":"2024-09-16T11:09:03.793526+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:08:06.827173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:08:06.827173+0000","last_clean_scrub_stamp":"2024-09-16T11:08:06.827173+0000","objects_scrubbed":0,"log_size":190,"log_dups_size":0,"ondisk_log_size":190,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00040617399999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":912,"num_read_kb":991,"num_write":288,"num_write_kb":407,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"34'232","reported_seq":697,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-16T11:09:02.822741+0000","last_change":"2024-09-16T11:08:53.883037+0000","last_active":"2024-09-16T11:09:02.822741+0000","last_peered":"2024-09-16T11:09:02.822741+0000","last_clean":"2024-09-16T11:09:02.822741+0000","last_became_active":"2024-09-16T11:08:07.853749+0000","last_became_peered":"2024-09-16T11:08:07.853749+0000","last_unstale":"2024-09-16T11:09:02.822741+0000","last_undegraded":"2024-09-16T11:09:02.822741+0000","last_fullsized":"2024-09-16T11:09:02.822741+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:08:06.827173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:08:06.827173+0000","last_clean_scrub_stamp":"2024-09-16T11:08:06.827173+0000","objects_scrubbed":0,"log_size":232,"log_dups_size":0,"ondisk_log_size":232,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00063776800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1659,"num_read_kb":2488,"num_write":313,"num_write_kb":1774,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":16005,"num_read_kb":49038,"num_write":3041,"num_write_kb":27918,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2483,"ondisk_log_size":2483,"up":64,"acting":64,"num_store_stats":3},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5,"ondisk_log_size":5,"up":16,"acting":16,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":56,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":54,"ondisk_log_size":54,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359738391,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301816,"kb_used_data":304,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94070024,"statfs":{"total":96636764160,"available":96327704576,"internally_reserved":0,"allocated":311296,"data_stored":120927,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 11:08:40 2024","interfaces":[{"interface":"back","average":{"1min":0.221,"5min":0.221,"15min":0.221},"min":{"1min":0.14599999999999999,"5min":0.14599999999999999,"15min":0.14599999999999999},"max":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"last":0.252},{"interface":"front","average":{"1min":0.23200000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"min":{"1min":0.17000000000000001,"5min":0.17000000000000001,"15min":0.17000000000000001},"max":{"1min":0.31,"5min":0.31,"15min":0.31},"last":0.26800000000000002}]},{"osd":1,"last update":"Mon Sep 16 11:08:40 2024","interfaces":[{"interface":"back","average":{"1min":0.23799999999999999,"5min":0.23799999999999999,"15min":0.23799999999999999},"min":{"1min":0.159,"5min":0.159,"15min":0.159},"max":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"last":0.28599999999999998},{"interface":"front","average":{"1min":0.252,"5min":0.252,"15min":0.252},"min":{"1min":0.184,"5min":0.184,"15min":0.184},"max":{"1min":0.376,"5min":0.376,"15min":0.376},"last":0.30399999999999999}]}]},{"osd":1,"up_from":8,"seq":34359738392,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":306564,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":305662,"kb_avail":94065276,"statfs":{"total":96636764160,"available":96322842624,"internally_reserved":0,"allocated":913408,"data_stored":712075,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":312998232},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 11:08:37 2024","interfaces":[{"interface":"back","average":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"min":{"1min":0.17399999999999999,"5min":0.17399999999999999,"15min":0.17399999999999999},"max":{"1min":0.376,"5min":0.376,"15min":0.376},"last":0.32800000000000001},{"interface":"front","average":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"min":{"1min":0.17599999999999999,"5min":0.17599999999999999,"15min":0.17599999999999999},"max":{"1min":0.376,"5min":0.376,"15min":0.376},"last":0.27400000000000002}]},{"osd":2,"last update":"Mon Sep 16 11:08:37 2024","interfaces":[{"interface":"back","average":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"min":{"1min":0.16600000000000001,"5min":0.16600000000000001,"15min":0.16600000000000001},"max":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"last":0.307},{"interface":"front","average":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"min":{"1min":0.191,"5min":0.191,"15min":0.191},"max":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"last":0.29199999999999998}]}]},{"osd":0,"up_from":8,"seq":34359738391,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":306556,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":305662,"kb_avail":94065284,"statfs":{"total":96636764160,"available":96322850816,"internally_reserved":0,"allocated":905216,"data_stored":711295,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":312998232},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Sep 16 11:08:40 2024","interfaces":[{"interface":"back","average":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"min":{"1min":0.14599999999999999,"5min":0.14599999999999999,"15min":0.14599999999999999},"max":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"last":0.35999999999999999},{"interface":"front","average":{"1min":0.31,"5min":0.31,"15min":0.31},"min":{"1min":0.16600000000000001,"5min":0.16600000000000001,"15min":0.16600000000000001},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.40999999999999998}]},{"osd":2,"last update":"Mon Sep 16 11:08:40 2024","interfaces":[{"interface":"back","average":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"min":{"1min":0.20300000000000001,"5min":0.20300000000000001,"15min":0.20300000000000001},"max":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"last":0.375},{"interface":"front","average":{"1min":0.309,"5min":0.309,"15min":0.309},"min":{"1min":0.186,"5min":0.186,"15min":0.186},"max":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"last":0.39300000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T11:09:18.671 INFO:tasks.ceph:pgid 3.1f last_scrub_stamp 2024-09-16T11:08:06.827173+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=8, tm_sec=6, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=8, tm_sec=55, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T11:09:18.671 INFO:tasks.ceph:pgid 3.1e last_scrub_stamp 2024-09-16T11:08:06.827173+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=8, tm_sec=6, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=8, tm_sec=55, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T11:09:18.671 INFO:tasks.ceph:pgid 3.1d last_scrub_stamp 2024-09-16T11:08:06.827173+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=8, tm_sec=6, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=8, tm_sec=55, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T11:09:18.671 INFO:tasks.ceph:pgid 3.1c last_scrub_stamp 2024-09-16T11:08:06.827173+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=8, tm_sec=6, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=8, tm_sec=55, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T11:09:18.671 INFO:tasks.ceph:pgid 3.1b last_scrub_stamp 2024-09-16T11:08:06.827173+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=8, tm_sec=6, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=8, tm_sec=55, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T11:09:18.672 INFO:tasks.ceph:pgid 3.1a last_scrub_stamp 2024-09-16T11:08:06.827173+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=8, tm_sec=6, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=8, tm_sec=55, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T11:09:18.672 INFO:tasks.ceph:pgid 3.19 last_scrub_stamp 2024-09-16T11:08:06.827173+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=8, tm_sec=6, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=8, tm_sec=55, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T11:09:18.672 INFO:tasks.ceph:pgid 3.18 last_scrub_stamp 2024-09-16T11:08:06.827173+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=8, tm_sec=6, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=8, tm_sec=55, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T11:09:18.672 INFO:tasks.ceph:pgid 3.17 last_scrub_stamp 2024-09-16T11:08:06.827173+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=8, tm_sec=6, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=8, tm_sec=55, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T11:09:18.672 INFO:tasks.ceph:pgid 3.16 last_scrub_stamp 2024-09-16T11:08:06.827173+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=8, tm_sec=6, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=8, tm_sec=55, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T11:09:18.672 INFO:tasks.ceph:pgid 3.15 last_scrub_stamp 2024-09-16T11:08:06.827173+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=8, tm_sec=6, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=8, tm_sec=55, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T11:09:18.672 INFO:tasks.ceph:pgid 3.14 last_scrub_stamp 2024-09-16T11:08:06.827173+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=8, tm_sec=6, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=8, tm_sec=55, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T11:09:18.672 INFO:tasks.ceph:pgid 3.6 last_scrub_stamp 2024-09-16T11:08:06.827173+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=8, tm_sec=6, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=8, tm_sec=55, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T11:09:18.673 INFO:tasks.ceph:pgid 3.7 last_scrub_stamp 2024-09-16T11:08:06.827173+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=8, tm_sec=6, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=8, tm_sec=55, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T11:09:18.673 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2024-09-16T11:07:43.427920+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=7, tm_sec=43, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=8, tm_sec=55, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T11:09:18.673 INFO:tasks.ceph:pgid 3.4 last_scrub_stamp 2024-09-16T11:08:06.827173+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=8, tm_sec=6, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=8, tm_sec=55, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T11:09:18.673 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2024-09-16T11:07:43.427920+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=7, tm_sec=43, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=8, tm_sec=55, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T11:09:18.673 INFO:tasks.ceph:pgid 3.5 last_scrub_stamp 2024-09-16T11:08:06.827173+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=8, tm_sec=6, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=8, tm_sec=55, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T11:09:18.673 INFO:tasks.ceph:pgid 3.0 last_scrub_stamp 2024-09-16T11:08:06.827173+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=8, tm_sec=6, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=8, tm_sec=55, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T11:09:18.673 INFO:tasks.ceph:pgid 3.8 last_scrub_stamp 2024-09-16T11:08:06.827173+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=8, tm_sec=6, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=8, tm_sec=55, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T11:09:18.673 INFO:tasks.ceph:pgid 3.9 last_scrub_stamp 2024-09-16T11:08:06.827173+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=8, tm_sec=6, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=8, tm_sec=55, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T11:09:18.673 INFO:tasks.ceph:pgid 3.a last_scrub_stamp 2024-09-16T11:08:06.827173+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=8, tm_sec=6, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=8, tm_sec=55, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T11:09:18.674 INFO:tasks.ceph:pgid 3.b last_scrub_stamp 2024-09-16T11:08:06.827173+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=8, tm_sec=6, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=8, tm_sec=55, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T11:09:18.674 INFO:tasks.ceph:pgid 3.d last_scrub_stamp 2024-09-16T11:08:06.827173+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=8, tm_sec=6, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=8, tm_sec=55, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T11:09:18.674 INFO:tasks.ceph:pgid 3.f last_scrub_stamp 2024-09-16T11:08:06.827173+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=8, tm_sec=6, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=8, tm_sec=55, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T11:09:18.674 INFO:tasks.ceph:pgid 3.10 last_scrub_stamp 2024-09-16T11:08:06.827173+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=8, tm_sec=6, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=8, tm_sec=55, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T11:09:18.674 INFO:tasks.ceph:pgid 3.12 last_scrub_stamp 2024-09-16T11:08:06.827173+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=8, tm_sec=6, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=8, tm_sec=55, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T11:09:18.674 INFO:tasks.ceph:pgid 3.13 last_scrub_stamp 2024-09-16T11:08:06.827173+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=8, tm_sec=6, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=8, tm_sec=55, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T11:09:18.674 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-09-16T11:09:38.675 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T11:09:38.952 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-16T11:09:38.952 INFO:teuthology.orchestra.run.smithi102.stderr:dumped all 2024-09-16T11:09:38.963 INFO:teuthology.orchestra.run.smithi102.stdout:{"pg_ready":true,"pg_map":{"version":95,"stamp":"2024-09-16T11:09:37.668487+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":16061,"num_read_kb":49084,"num_write":3122,"num_write_kb":29026,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2542,"ondisk_log_size":2542,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":914936,"kb_used_data":2080,"kb_used_omap":4,"kb_used_meta":912827,"kb_avail":282200584,"statfs":{"total":289910292480,"available":288973398016,"internally_reserved":0,"allocated":2129920,"data_stored":1544297,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":5112,"internal_metadata":934734856},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002101"},"pg_stats":[{"pgid":"3.1f","version":"35'35","reported_seq":218,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-16T11:09:03.794339+0000","last_change":"2024-09-16T11:08:53.899921+0000","last_active":"2024-09-16T11:09:03.794339+0000","last_peered":"2024-09-16T11:09:03.794339+0000","last_clean":"2024-09-16T11:09:03.794339+0000","last_became_active":"2024-09-16T11:08:07.859102+0000","last_became_peered":"2024-09-16T11:08:07.859102+0000","last_unstale":"2024-09-16T11:09:03.794339+0000","last_undegraded":"2024-09-16T11:09:03.794339+0000","last_fullsized":"2024-09-16T11:09:03.794339+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:08:06.827173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:08:06.827173+0000","last_clean_scrub_stamp":"2024-09-16T11:08:06.827173+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00095524,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":130,"num_read_kb":1007,"num_write":36,"num_write_kb":861,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"35'187","reported_seq":868,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-16T11:09:24.625883+0000","last_change":"2024-09-16T11:09:24.625856+0000","last_active":"2024-09-16T11:09:24.625883+0000","last_peered":"2024-09-16T11:09:24.625883+0000","last_clean":"2024-09-16T11:09:24.625883+0000","last_became_active":"2024-09-16T11:08:07.856092+0000","last_became_peered":"2024-09-16T11:08:07.856092+0000","last_unstale":"2024-09-16T11:09:24.625883+0000","last_undegraded":"2024-09-16T11:09:24.625883+0000","last_fullsized":"2024-09-16T11:09:24.625883+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"35'187","last_scrub_stamp":"2024-09-16T11:09:24.625766+0000","last_deep_scrub":"35'187","last_deep_scrub_stamp":"2024-09-16T11:09:24.625766+0000","last_clean_scrub_stamp":"2024-09-16T11:09:24.625766+0000","objects_scrubbed":0,"log_size":187,"log_dups_size":0,"ondisk_log_size":187,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:17:47.522929+0000","scrub_duration":0.0013864529999999999,"objects_trimmed":0,"snaptrim_duration":0.001038806,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1989,"num_read_kb":2691,"num_write":225,"num_write_kb":434,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1d","version":"35'287","reported_seq":373,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-16T11:09:26.581258+0000","last_change":"2024-09-16T11:09:26.581231+0000","last_active":"2024-09-16T11:09:26.581258+0000","last_peered":"2024-09-16T11:09:26.581258+0000","last_clean":"2024-09-16T11:09:26.581258+0000","last_became_active":"2024-09-16T11:08:07.847513+0000","last_became_peered":"2024-09-16T11:08:07.847513+0000","last_unstale":"2024-09-16T11:09:26.581258+0000","last_undegraded":"2024-09-16T11:09:26.581258+0000","last_fullsized":"2024-09-16T11:09:26.581258+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"35'287","last_scrub_stamp":"2024-09-16T11:09:26.581133+0000","last_deep_scrub":"35'287","last_deep_scrub_stamp":"2024-09-16T11:09:26.581133+0000","last_clean_scrub_stamp":"2024-09-16T11:09:26.581133+0000","objects_scrubbed":0,"log_size":287,"log_dups_size":0,"ondisk_log_size":287,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:10:04.869239+0000","scrub_duration":0.001488461,"objects_trimmed":0,"snaptrim_duration":0.000667575,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1265,"num_read_kb":1036,"num_write":459,"num_write_kb":602,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"35'36","reported_seq":237,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-16T11:09:02.822249+0000","last_change":"2024-09-16T11:08:53.883362+0000","last_active":"2024-09-16T11:09:02.822249+0000","last_peered":"2024-09-16T11:09:02.822249+0000","last_clean":"2024-09-16T11:09:02.822249+0000","last_became_active":"2024-09-16T11:08:07.855430+0000","last_became_peered":"2024-09-16T11:08:07.855430+0000","last_unstale":"2024-09-16T11:09:02.822249+0000","last_undegraded":"2024-09-16T11:09:02.822249+0000","last_fullsized":"2024-09-16T11:09:02.822249+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:08:06.827173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:08:06.827173+0000","last_clean_scrub_stamp":"2024-09-16T11:08:06.827173+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097256700000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":177,"num_read_kb":260,"num_write":47,"num_write_kb":566,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"34'165","reported_seq":859,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-16T11:09:03.793819+0000","last_change":"2024-09-16T11:08:53.899200+0000","last_active":"2024-09-16T11:09:03.793819+0000","last_peered":"2024-09-16T11:09:03.793819+0000","last_clean":"2024-09-16T11:09:03.793819+0000","last_became_active":"2024-09-16T11:08:07.856827+0000","last_became_peered":"2024-09-16T11:08:07.856827+0000","last_unstale":"2024-09-16T11:09:03.793819+0000","last_undegraded":"2024-09-16T11:09:03.793819+0000","last_fullsized":"2024-09-16T11:09:03.793819+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:08:06.827173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:08:06.827173+0000","last_clean_scrub_stamp":"2024-09-16T11:08:06.827173+0000","objects_scrubbed":0,"log_size":165,"log_dups_size":0,"ondisk_log_size":165,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081398199999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1407,"num_read_kb":2244,"num_write":140,"num_write_kb":887,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"28'13","reported_seq":399,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-16T11:09:02.822209+0000","last_change":"2024-09-16T11:08:53.883198+0000","last_active":"2024-09-16T11:09:02.822209+0000","last_peered":"2024-09-16T11:09:02.822209+0000","last_clean":"2024-09-16T11:09:02.822209+0000","last_became_active":"2024-09-16T11:08:07.853372+0000","last_became_peered":"2024-09-16T11:08:07.853372+0000","last_unstale":"2024-09-16T11:09:02.822209+0000","last_undegraded":"2024-09-16T11:09:02.822209+0000","last_fullsized":"2024-09-16T11:09:02.822209+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:08:06.827173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:08:06.827173+0000","last_clean_scrub_stamp":"2024-09-16T11:08:06.827173+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00082456000000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":325,"num_read_kb":343,"num_write":13,"num_write_kb":81,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.19","version":"35'47","reported_seq":302,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-16T11:09:02.822105+0000","last_change":"2024-09-16T11:08:53.882678+0000","last_active":"2024-09-16T11:09:02.822105+0000","last_peered":"2024-09-16T11:09:02.822105+0000","last_clean":"2024-09-16T11:09:02.822105+0000","last_became_active":"2024-09-16T11:08:07.852874+0000","last_became_peered":"2024-09-16T11:08:07.852874+0000","last_unstale":"2024-09-16T11:09:02.822105+0000","last_undegraded":"2024-09-16T11:09:02.822105+0000","last_fullsized":"2024-09-16T11:09:02.822105+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:08:06.827173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:08:06.827173+0000","last_clean_scrub_stamp":"2024-09-16T11:08:06.827173+0000","objects_scrubbed":0,"log_size":47,"log_dups_size":0,"ondisk_log_size":47,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00052581399999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":214,"num_read_kb":667,"num_write":46,"num_write_kb":746,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"35'18","reported_seq":82,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-16T11:09:03.588435+0000","last_change":"2024-09-16T11:08:53.887745+0000","last_active":"2024-09-16T11:09:03.588435+0000","last_peered":"2024-09-16T11:09:03.588435+0000","last_clean":"2024-09-16T11:09:03.588435+0000","last_became_active":"2024-09-16T11:08:07.848434+0000","last_became_peered":"2024-09-16T11:08:07.848434+0000","last_unstale":"2024-09-16T11:09:03.588435+0000","last_undegraded":"2024-09-16T11:09:03.588435+0000","last_fullsized":"2024-09-16T11:09:03.588435+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:08:06.827173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:08:06.827173+0000","last_clean_scrub_stamp":"2024-09-16T11:08:06.827173+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043492900000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":18,"num_write_kb":60,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.17","version":"31'25","reported_seq":124,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-16T11:09:23.766121+0000","last_change":"2024-09-16T11:09:23.766090+0000","last_active":"2024-09-16T11:09:23.766121+0000","last_peered":"2024-09-16T11:09:23.766121+0000","last_clean":"2024-09-16T11:09:23.766121+0000","last_became_active":"2024-09-16T11:08:07.859097+0000","last_became_peered":"2024-09-16T11:08:07.859097+0000","last_unstale":"2024-09-16T11:09:23.766121+0000","last_undegraded":"2024-09-16T11:09:23.766121+0000","last_fullsized":"2024-09-16T11:09:23.766121+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'25","last_scrub_stamp":"2024-09-16T11:09:23.765992+0000","last_deep_scrub":"31'25","last_deep_scrub_stamp":"2024-09-16T11:09:23.765992+0000","last_clean_scrub_stamp":"2024-09-16T11:09:23.765992+0000","objects_scrubbed":0,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:10:02.054098+0000","scrub_duration":0.0014903340000000001,"objects_trimmed":0,"snaptrim_duration":0.001161507,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":30,"num_read_kb":66,"num_write":24,"num_write_kb":325,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"32'25","reported_seq":153,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-16T11:09:18.566133+0000","last_change":"2024-09-16T11:09:18.566112+0000","last_active":"2024-09-16T11:09:18.566133+0000","last_peered":"2024-09-16T11:09:18.566133+0000","last_clean":"2024-09-16T11:09:18.566133+0000","last_became_active":"2024-09-16T11:08:07.856007+0000","last_became_peered":"2024-09-16T11:08:07.856007+0000","last_unstale":"2024-09-16T11:09:18.566133+0000","last_undegraded":"2024-09-16T11:09:18.566133+0000","last_fullsized":"2024-09-16T11:09:18.566133+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'25","last_scrub_stamp":"2024-09-16T11:09:18.566035+0000","last_deep_scrub":"32'25","last_deep_scrub_stamp":"2024-09-16T11:09:18.566035+0000","last_clean_scrub_stamp":"2024-09-16T11:09:18.566035+0000","objects_scrubbed":0,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:07:14.883915+0000","scrub_duration":0.0011610380000000001,"objects_trimmed":0,"snaptrim_duration":0.0011111560000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":66,"num_read_kb":1713,"num_write":38,"num_write_kb":1539,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"33'31","reported_seq":185,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-16T11:09:32.644690+0000","last_change":"2024-09-16T11:09:32.644664+0000","last_active":"2024-09-16T11:09:32.644690+0000","last_peered":"2024-09-16T11:09:32.644690+0000","last_clean":"2024-09-16T11:09:32.644690+0000","last_became_active":"2024-09-16T11:08:07.858375+0000","last_became_peered":"2024-09-16T11:08:07.858375+0000","last_unstale":"2024-09-16T11:09:32.644690+0000","last_undegraded":"2024-09-16T11:09:32.644690+0000","last_fullsized":"2024-09-16T11:09:32.644690+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'31","last_scrub_stamp":"2024-09-16T11:09:32.644569+0000","last_deep_scrub":"33'31","last_deep_scrub_stamp":"2024-09-16T11:09:32.644569+0000","last_clean_scrub_stamp":"2024-09-16T11:09:32.644569+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:36:18.603929+0000","scrub_duration":0.001451052,"objects_trimmed":0,"snaptrim_duration":0.00079626899999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":92,"num_read_kb":770,"num_write":36,"num_write_kb":1093,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.14","version":"35'141","reported_seq":308,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-16T11:09:02.822320+0000","last_change":"2024-09-16T11:08:53.883413+0000","last_active":"2024-09-16T11:09:02.822320+0000","last_peered":"2024-09-16T11:09:02.822320+0000","last_clean":"2024-09-16T11:09:02.822320+0000","last_became_active":"2024-09-16T11:08:07.853189+0000","last_became_peered":"2024-09-16T11:08:07.853189+0000","last_unstale":"2024-09-16T11:09:02.822320+0000","last_undegraded":"2024-09-16T11:09:02.822320+0000","last_fullsized":"2024-09-16T11:09:02.822320+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:08:06.827173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:08:06.827173+0000","last_clean_scrub_stamp":"2024-09-16T11:08:06.827173+0000","objects_scrubbed":0,"log_size":141,"log_dups_size":0,"ondisk_log_size":141,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00087565099999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":523,"num_read_kb":930,"num_write":189,"num_write_kb":943,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":48,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-16T11:09:03.797856+0000","last_change":"2024-09-16T11:09:03.797818+0000","last_active":"2024-09-16T11:09:03.797856+0000","last_peered":"2024-09-16T11:09:03.797856+0000","last_clean":"2024-09-16T11:09:03.797856+0000","last_became_active":"2024-09-16T11:07:44.446646+0000","last_became_peered":"2024-09-16T11:07:44.446646+0000","last_unstale":"2024-09-16T11:09:03.797856+0000","last_undegraded":"2024-09-16T11:09:03.797856+0000","last_fullsized":"2024-09-16T11:09:03.797856+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:09:03.797714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:09:03.797714+0000","last_clean_scrub_stamp":"2024-09-16T11:09:03.797714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:29:49.984164+0000","scrub_duration":0.0062555400000000004,"objects_trimmed":0,"snaptrim_duration":0.000337344,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.6","version":"35'83","reported_seq":163,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-16T11:09:21.742649+0000","last_change":"2024-09-16T11:09:21.742613+0000","last_active":"2024-09-16T11:09:21.742649+0000","last_peered":"2024-09-16T11:09:21.742649+0000","last_clean":"2024-09-16T11:09:21.742649+0000","last_became_active":"2024-09-16T11:08:07.858208+0000","last_became_peered":"2024-09-16T11:08:07.858208+0000","last_unstale":"2024-09-16T11:09:21.742649+0000","last_undegraded":"2024-09-16T11:09:21.742649+0000","last_fullsized":"2024-09-16T11:09:21.742649+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"35'83","last_scrub_stamp":"2024-09-16T11:09:21.742515+0000","last_deep_scrub":"35'83","last_deep_scrub_stamp":"2024-09-16T11:09:21.742515+0000","last_clean_scrub_stamp":"2024-09-16T11:09:21.742515+0000","objects_scrubbed":0,"log_size":83,"log_dups_size":0,"ondisk_log_size":83,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:17:44.639678+0000","scrub_duration":0.0015147889999999999,"objects_trimmed":0,"snaptrim_duration":0.00098881699999999991,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":259,"num_read_kb":213,"num_write":114,"num_write_kb":675,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":48,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-16T11:09:07.770754+0000","last_change":"2024-09-16T11:09:07.770730+0000","last_active":"2024-09-16T11:09:07.770754+0000","last_peered":"2024-09-16T11:09:07.770754+0000","last_clean":"2024-09-16T11:09:07.770754+0000","last_became_active":"2024-09-16T11:07:44.446406+0000","last_became_peered":"2024-09-16T11:07:44.446406+0000","last_unstale":"2024-09-16T11:09:07.770754+0000","last_undegraded":"2024-09-16T11:09:07.770754+0000","last_fullsized":"2024-09-16T11:09:07.770754+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:09:07.770642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:09:07.770642+0000","last_clean_scrub_stamp":"2024-09-16T11:09:07.770642+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:29:11.031172+0000","scrub_duration":0.001319305,"objects_trimmed":0,"snaptrim_duration":0.00062301399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"35'44","reported_seq":342,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-16T11:09:21.594392+0000","last_change":"2024-09-16T11:09:21.594367+0000","last_active":"2024-09-16T11:09:21.594392+0000","last_peered":"2024-09-16T11:09:21.594392+0000","last_clean":"2024-09-16T11:09:21.594392+0000","last_became_active":"2024-09-16T11:08:07.856393+0000","last_became_peered":"2024-09-16T11:08:07.856393+0000","last_unstale":"2024-09-16T11:09:21.594392+0000","last_undegraded":"2024-09-16T11:09:21.594392+0000","last_fullsized":"2024-09-16T11:09:21.594392+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"35'44","last_scrub_stamp":"2024-09-16T11:09:21.594260+0000","last_deep_scrub":"35'44","last_deep_scrub_stamp":"2024-09-16T11:09:21.594260+0000","last_clean_scrub_stamp":"2024-09-16T11:09:21.594260+0000","objects_scrubbed":0,"log_size":44,"log_dups_size":0,"ondisk_log_size":44,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:49:29.267016+0000","scrub_duration":0.0013851639999999999,"objects_trimmed":0,"snaptrim_duration":0.00064218800000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":241,"num_read_kb":2845,"num_write":55,"num_write_kb":2044,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":48,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-16T11:09:17.757451+0000","last_change":"2024-09-16T11:09:17.757404+0000","last_active":"2024-09-16T11:09:17.757451+0000","last_peered":"2024-09-16T11:09:17.757451+0000","last_clean":"2024-09-16T11:09:17.757451+0000","last_became_active":"2024-09-16T11:07:44.446498+0000","last_became_peered":"2024-09-16T11:07:44.446498+0000","last_unstale":"2024-09-16T11:09:17.757451+0000","last_undegraded":"2024-09-16T11:09:17.757451+0000","last_fullsized":"2024-09-16T11:09:17.757451+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:09:17.757291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:09:17.757291+0000","last_clean_scrub_stamp":"2024-09-16T11:09:17.757291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:17:40.654454+0000","scrub_duration":0.001523022,"objects_trimmed":0,"snaptrim_duration":0.00084251400000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"32'67","reported_seq":291,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-16T11:09:19.738684+0000","last_change":"2024-09-16T11:09:19.738637+0000","last_active":"2024-09-16T11:09:19.738684+0000","last_peered":"2024-09-16T11:09:19.738684+0000","last_clean":"2024-09-16T11:09:19.738684+0000","last_became_active":"2024-09-16T11:08:07.853470+0000","last_became_peered":"2024-09-16T11:08:07.853470+0000","last_unstale":"2024-09-16T11:09:19.738684+0000","last_undegraded":"2024-09-16T11:09:19.738684+0000","last_fullsized":"2024-09-16T11:09:19.738684+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'67","last_scrub_stamp":"2024-09-16T11:09:19.738524+0000","last_deep_scrub":"32'67","last_deep_scrub_stamp":"2024-09-16T11:09:19.738524+0000","last_clean_scrub_stamp":"2024-09-16T11:09:19.738524+0000","objects_scrubbed":0,"log_size":67,"log_dups_size":0,"ondisk_log_size":67,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:09:58.026631+0000","scrub_duration":0.0016065560000000001,"objects_trimmed":0,"snaptrim_duration":0.001033672,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":360,"num_read_kb":1468,"num_write":100,"num_write_kb":888,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":48,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-16T11:09:14.743581+0000","last_change":"2024-09-16T11:09:14.743549+0000","last_active":"2024-09-16T11:09:14.743581+0000","last_peered":"2024-09-16T11:09:14.743581+0000","last_clean":"2024-09-16T11:09:14.743581+0000","last_became_active":"2024-09-16T11:07:44.446413+0000","last_became_peered":"2024-09-16T11:07:44.446413+0000","last_unstale":"2024-09-16T11:09:14.743581+0000","last_undegraded":"2024-09-16T11:09:14.743581+0000","last_fullsized":"2024-09-16T11:09:14.743581+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:09:14.743448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:09:14.743448+0000","last_clean_scrub_stamp":"2024-09-16T11:09:14.743448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:49:22.416205+0000","scrub_duration":0.0014080869999999999,"objects_trimmed":0,"snaptrim_duration":0.00059042800000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"34'78","reported_seq":502,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-16T11:09:14.635358+0000","last_change":"2024-09-16T11:09:14.635326+0000","last_active":"2024-09-16T11:09:14.635358+0000","last_peered":"2024-09-16T11:09:14.635358+0000","last_clean":"2024-09-16T11:09:14.635358+0000","last_became_active":"2024-09-16T11:08:07.855280+0000","last_became_peered":"2024-09-16T11:08:07.855280+0000","last_unstale":"2024-09-16T11:09:14.635358+0000","last_undegraded":"2024-09-16T11:09:14.635358+0000","last_fullsized":"2024-09-16T11:09:14.635358+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'78","last_scrub_stamp":"2024-09-16T11:09:14.635225+0000","last_deep_scrub":"34'78","last_deep_scrub_stamp":"2024-09-16T11:09:14.635225+0000","last_clean_scrub_stamp":"2024-09-16T11:09:14.635225+0000","objects_scrubbed":0,"log_size":78,"log_dups_size":0,"ondisk_log_size":78,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:29:17.895755+0000","scrub_duration":0.0015130829999999999,"objects_trimmed":0,"snaptrim_duration":0.00040932799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":396,"num_read_kb":2349,"num_write":79,"num_write_kb":1069,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"35'86","reported_seq":288,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-16T11:09:07.764657+0000","last_change":"2024-09-16T11:09:07.764626+0000","last_active":"2024-09-16T11:09:07.764657+0000","last_peered":"2024-09-16T11:09:07.764657+0000","last_clean":"2024-09-16T11:09:07.764657+0000","last_became_active":"2024-09-16T11:08:07.847550+0000","last_became_peered":"2024-09-16T11:08:07.847550+0000","last_unstale":"2024-09-16T11:09:07.764657+0000","last_undegraded":"2024-09-16T11:09:07.764657+0000","last_fullsized":"2024-09-16T11:09:07.764657+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"35'86","last_scrub_stamp":"2024-09-16T11:09:07.764522+0000","last_deep_scrub":"35'86","last_deep_scrub_stamp":"2024-09-16T11:09:07.764522+0000","last_clean_scrub_stamp":"2024-09-16T11:09:07.764522+0000","objects_scrubbed":1,"log_size":86,"log_dups_size":0,"ondisk_log_size":86,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:29:53.950973+0000","scrub_duration":0.0017137980000000001,"objects_trimmed":0,"snaptrim_duration":0.000875419,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":164,"num_read_kb":968,"num_write":116,"num_write_kb":2579,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"35'3","reported_seq":50,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-16T11:08:58.393319+0000","last_change":"2024-09-16T11:08:57.637442+0000","last_active":"2024-09-16T11:08:58.393319+0000","last_peered":"2024-09-16T11:08:58.393319+0000","last_clean":"2024-09-16T11:08:58.393319+0000","last_became_active":"2024-09-16T11:07:44.449069+0000","last_became_peered":"2024-09-16T11:07:44.449069+0000","last_unstale":"2024-09-16T11:08:58.393319+0000","last_undegraded":"2024-09-16T11:08:58.393319+0000","last_fullsized":"2024-09-16T11:08:58.393319+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"35'3","last_scrub_stamp":"2024-09-16T11:08:57.637376+0000","last_deep_scrub":"35'3","last_deep_scrub_stamp":"2024-09-16T11:08:57.637376+0000","last_clean_scrub_stamp":"2024-09-16T11:08:57.637376+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:59:04.639271+0000","scrub_duration":0.0023648480000000001,"objects_trimmed":0,"snaptrim_duration":0.00029305100000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.0","version":"34'47","reported_seq":220,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-16T11:09:02.822147+0000","last_change":"2024-09-16T11:08:53.882919+0000","last_active":"2024-09-16T11:09:02.822147+0000","last_peered":"2024-09-16T11:09:02.822147+0000","last_clean":"2024-09-16T11:09:02.822147+0000","last_became_active":"2024-09-16T11:08:07.855258+0000","last_became_peered":"2024-09-16T11:08:07.855258+0000","last_unstale":"2024-09-16T11:09:02.822147+0000","last_undegraded":"2024-09-16T11:09:02.822147+0000","last_fullsized":"2024-09-16T11:09:02.822147+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:08:06.827173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:08:06.827173+0000","last_clean_scrub_stamp":"2024-09-16T11:08:06.827173+0000","objects_scrubbed":0,"log_size":47,"log_dups_size":0,"ondisk_log_size":47,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00075381500000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":141,"num_read_kb":1673,"num_write":53,"num_write_kb":1263,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":47,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-16T11:09:06.562694+0000","last_change":"2024-09-16T11:09:06.562665+0000","last_active":"2024-09-16T11:09:06.562694+0000","last_peered":"2024-09-16T11:09:06.562694+0000","last_clean":"2024-09-16T11:09:06.562694+0000","last_became_active":"2024-09-16T11:07:44.447533+0000","last_became_peered":"2024-09-16T11:07:44.447533+0000","last_unstale":"2024-09-16T11:09:06.562694+0000","last_undegraded":"2024-09-16T11:09:06.562694+0000","last_fullsized":"2024-09-16T11:09:06.562694+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:09:06.562572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:09:06.562572+0000","last_clean_scrub_stamp":"2024-09-16T11:09:06.562572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:43:14.347543+0000","scrub_duration":0.0013831399999999999,"objects_trimmed":0,"snaptrim_duration":0.000230378,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"33'30","reported_seq":185,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-16T11:09:03.804035+0000","last_change":"2024-09-16T11:09:03.804009+0000","last_active":"2024-09-16T11:09:03.804035+0000","last_peered":"2024-09-16T11:09:03.804035+0000","last_clean":"2024-09-16T11:09:03.804035+0000","last_became_active":"2024-09-16T11:08:07.847955+0000","last_became_peered":"2024-09-16T11:08:07.847955+0000","last_unstale":"2024-09-16T11:09:03.804035+0000","last_undegraded":"2024-09-16T11:09:03.804035+0000","last_fullsized":"2024-09-16T11:09:03.804035+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'30","last_scrub_stamp":"2024-09-16T11:09:03.803902+0000","last_deep_scrub":"33'30","last_deep_scrub_stamp":"2024-09-16T11:09:03.803902+0000","last_clean_scrub_stamp":"2024-09-16T11:09:03.803902+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:43:11.588873+0000","scrub_duration":0.001499385,"objects_trimmed":0,"snaptrim_duration":0.00029393900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":87,"num_read_kb":1164,"num_write":31,"num_write_kb":740,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":47,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-16T11:09:01.573333+0000","last_change":"2024-09-16T11:09:01.573294+0000","last_active":"2024-09-16T11:09:01.573333+0000","last_peered":"2024-09-16T11:09:01.573333+0000","last_clean":"2024-09-16T11:09:01.573333+0000","last_became_active":"2024-09-16T11:07:44.448153+0000","last_became_peered":"2024-09-16T11:07:44.448153+0000","last_unstale":"2024-09-16T11:09:01.573333+0000","last_undegraded":"2024-09-16T11:09:01.573333+0000","last_fullsized":"2024-09-16T11:09:01.573333+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:09:01.573204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:09:01.573204+0000","last_clean_scrub_stamp":"2024-09-16T11:09:01.573204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:59:08.575099+0000","scrub_duration":0.001347532,"objects_trimmed":0,"snaptrim_duration":0.00024792500000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"34'46","reported_seq":296,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-16T11:09:02.822765+0000","last_change":"2024-09-16T11:08:59.685510+0000","last_active":"2024-09-16T11:09:02.822765+0000","last_peered":"2024-09-16T11:09:02.822765+0000","last_clean":"2024-09-16T11:09:02.822765+0000","last_became_active":"2024-09-16T11:08:07.853307+0000","last_became_peered":"2024-09-16T11:08:07.853307+0000","last_unstale":"2024-09-16T11:09:02.822765+0000","last_undegraded":"2024-09-16T11:09:02.822765+0000","last_fullsized":"2024-09-16T11:09:02.822765+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'46","last_scrub_stamp":"2024-09-16T11:08:59.685380+0000","last_deep_scrub":"34'46","last_deep_scrub_stamp":"2024-09-16T11:08:59.685380+0000","last_clean_scrub_stamp":"2024-09-16T11:08:59.685380+0000","objects_scrubbed":0,"log_size":46,"log_dups_size":0,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:43:07.470351+0000","scrub_duration":0.0015725050000000001,"objects_trimmed":0,"snaptrim_duration":0.0011803549999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":203,"num_read_kb":1989,"num_write":39,"num_write_kb":217,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"37'2","reported_seq":50,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-16T11:09:11.724730+0000","last_change":"2024-09-16T11:09:11.724697+0000","last_active":"2024-09-16T11:09:11.724730+0000","last_peered":"2024-09-16T11:09:11.724730+0000","last_clean":"2024-09-16T11:09:11.724730+0000","last_became_active":"2024-09-16T11:07:44.446367+0000","last_became_peered":"2024-09-16T11:07:44.446367+0000","last_unstale":"2024-09-16T11:09:11.724730+0000","last_undegraded":"2024-09-16T11:09:11.724730+0000","last_fullsized":"2024-09-16T11:09:11.724730+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2","last_scrub_stamp":"2024-09-16T11:09:11.724600+0000","last_deep_scrub":"37'2","last_deep_scrub_stamp":"2024-09-16T11:09:11.724600+0000","last_clean_scrub_stamp":"2024-09-16T11:09:11.724600+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:07:08.042481+0000","scrub_duration":0.0026358029999999999,"objects_trimmed":0,"snaptrim_duration":0.00038771200000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"36'54","reported_seq":115,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-16T11:09:02.760758+0000","last_change":"2024-09-16T11:09:02.760678+0000","last_active":"2024-09-16T11:09:02.760758+0000","last_peered":"2024-09-16T11:09:02.760758+0000","last_clean":"2024-09-16T11:09:02.760758+0000","last_became_active":"2024-09-16T11:07:39.420564+0000","last_became_peered":"2024-09-16T11:07:39.420564+0000","last_unstale":"2024-09-16T11:09:02.760758+0000","last_undegraded":"2024-09-16T11:09:02.760758+0000","last_fullsized":"2024-09-16T11:09:02.760758+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'54","last_scrub_stamp":"2024-09-16T11:09:02.760577+0000","last_deep_scrub":"36'54","last_deep_scrub_stamp":"2024-09-16T11:09:02.760577+0000","last_clean_scrub_stamp":"2024-09-16T11:09:02.760577+0000","objects_scrubbed":2,"log_size":54,"log_dups_size":0,"ondisk_log_size":54,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:11:57.859673+0000","scrub_duration":0.0048002319999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":56,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"33'41","reported_seq":463,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-16T11:09:03.588471+0000","last_change":"2024-09-16T11:08:53.888661+0000","last_active":"2024-09-16T11:09:03.588471+0000","last_peered":"2024-09-16T11:09:03.588471+0000","last_clean":"2024-09-16T11:09:03.588471+0000","last_became_active":"2024-09-16T11:08:07.857280+0000","last_became_peered":"2024-09-16T11:08:07.857280+0000","last_unstale":"2024-09-16T11:09:03.588471+0000","last_undegraded":"2024-09-16T11:09:03.588471+0000","last_fullsized":"2024-09-16T11:09:03.588471+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:08:06.827173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:08:06.827173+0000","last_clean_scrub_stamp":"2024-09-16T11:08:06.827173+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001178323,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":381,"num_read_kb":3183,"num_write":40,"num_write_kb":936,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.9","version":"35'18","reported_seq":81,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-16T11:09:03.793436+0000","last_change":"2024-09-16T11:08:53.898314+0000","last_active":"2024-09-16T11:09:03.793436+0000","last_peered":"2024-09-16T11:09:03.793436+0000","last_clean":"2024-09-16T11:09:03.793436+0000","last_became_active":"2024-09-16T11:08:07.850403+0000","last_became_peered":"2024-09-16T11:08:07.850403+0000","last_unstale":"2024-09-16T11:09:03.793436+0000","last_undegraded":"2024-09-16T11:09:03.793436+0000","last_fullsized":"2024-09-16T11:09:03.793436+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:08:06.827173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:08:06.827173+0000","last_clean_scrub_stamp":"2024-09-16T11:08:06.827173+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00025477399999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":55,"num_write":21,"num_write_kb":629,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.a","version":"34'51","reported_seq":389,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-16T11:09:15.758425+0000","last_change":"2024-09-16T11:09:15.758394+0000","last_active":"2024-09-16T11:09:15.758425+0000","last_peered":"2024-09-16T11:09:15.758425+0000","last_clean":"2024-09-16T11:09:15.758425+0000","last_became_active":"2024-09-16T11:08:07.857407+0000","last_became_peered":"2024-09-16T11:08:07.857407+0000","last_unstale":"2024-09-16T11:09:15.758425+0000","last_undegraded":"2024-09-16T11:09:15.758425+0000","last_fullsized":"2024-09-16T11:09:15.758425+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'51","last_scrub_stamp":"2024-09-16T11:09:15.758293+0000","last_deep_scrub":"34'51","last_deep_scrub_stamp":"2024-09-16T11:09:15.758293+0000","last_clean_scrub_stamp":"2024-09-16T11:09:15.758293+0000","objects_scrubbed":1,"log_size":51,"log_dups_size":0,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:07:12.076173+0000","scrub_duration":0.0017331250000000001,"objects_trimmed":0,"snaptrim_duration":0.00055050900000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":289,"num_read_kb":3215,"num_write":55,"num_write_kb":1438,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"34'104","reported_seq":901,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-16T11:09:28.657036+0000","last_change":"2024-09-16T11:09:28.656995+0000","last_active":"2024-09-16T11:09:28.657036+0000","last_peered":"2024-09-16T11:09:28.657036+0000","last_clean":"2024-09-16T11:09:28.657036+0000","last_became_active":"2024-09-16T11:08:07.855380+0000","last_became_peered":"2024-09-16T11:08:07.855380+0000","last_unstale":"2024-09-16T11:09:28.657036+0000","last_undegraded":"2024-09-16T11:09:28.657036+0000","last_fullsized":"2024-09-16T11:09:28.657036+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'104","last_scrub_stamp":"2024-09-16T11:09:28.656884+0000","last_deep_scrub":"34'104","last_deep_scrub_stamp":"2024-09-16T11:09:28.656884+0000","last_clean_scrub_stamp":"2024-09-16T11:09:28.656884+0000","objects_scrubbed":0,"log_size":104,"log_dups_size":0,"ondisk_log_size":104,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:36:14.616243+0000","scrub_duration":0.0014366660000000001,"objects_trimmed":0,"snaptrim_duration":0.001457519,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1288,"num_read_kb":4433,"num_write":108,"num_write_kb":1673,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"35'73","reported_seq":426,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-16T11:09:11.792421+0000","last_change":"2024-09-16T11:09:11.792370+0000","last_active":"2024-09-16T11:09:11.792421+0000","last_peered":"2024-09-16T11:09:11.792421+0000","last_clean":"2024-09-16T11:09:11.792421+0000","last_became_active":"2024-09-16T11:08:07.859197+0000","last_became_peered":"2024-09-16T11:08:07.859197+0000","last_unstale":"2024-09-16T11:09:11.792421+0000","last_undegraded":"2024-09-16T11:09:11.792421+0000","last_fullsized":"2024-09-16T11:09:11.792421+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"35'73","last_scrub_stamp":"2024-09-16T11:09:11.792260+0000","last_deep_scrub":"35'73","last_deep_scrub_stamp":"2024-09-16T11:09:11.792260+0000","last_clean_scrub_stamp":"2024-09-16T11:09:11.792260+0000","objects_scrubbed":0,"log_size":73,"log_dups_size":0,"ondisk_log_size":73,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:29:15.052789+0000","scrub_duration":0.001460004,"objects_trimmed":0,"snaptrim_duration":0.00087272100000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":617,"num_read_kb":1509,"num_write":86,"num_write_kb":959,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.d","version":"34'78","reported_seq":483,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-16T11:09:29.637726+0000","last_change":"2024-09-16T11:09:29.637694+0000","last_active":"2024-09-16T11:09:29.637726+0000","last_peered":"2024-09-16T11:09:29.637726+0000","last_clean":"2024-09-16T11:09:29.637726+0000","last_became_active":"2024-09-16T11:08:07.853865+0000","last_became_peered":"2024-09-16T11:08:07.853865+0000","last_unstale":"2024-09-16T11:09:29.637726+0000","last_undegraded":"2024-09-16T11:09:29.637726+0000","last_fullsized":"2024-09-16T11:09:29.637726+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'78","last_scrub_stamp":"2024-09-16T11:09:29.637604+0000","last_deep_scrub":"34'78","last_deep_scrub_stamp":"2024-09-16T11:09:29.637604+0000","last_clean_scrub_stamp":"2024-09-16T11:09:29.637604+0000","objects_scrubbed":0,"log_size":78,"log_dups_size":0,"ondisk_log_size":78,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:18:23.814335+0000","scrub_duration":0.0013750100000000001,"objects_trimmed":0,"snaptrim_duration":0.001054501,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":785,"num_read_kb":1332,"num_write":70,"num_write_kb":661,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.e","version":"34'55","reported_seq":340,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-16T11:09:10.614689+0000","last_change":"2024-09-16T11:09:10.614661+0000","last_active":"2024-09-16T11:09:10.614689+0000","last_peered":"2024-09-16T11:09:10.614689+0000","last_clean":"2024-09-16T11:09:10.614689+0000","last_became_active":"2024-09-16T11:08:07.847486+0000","last_became_peered":"2024-09-16T11:08:07.847486+0000","last_unstale":"2024-09-16T11:09:10.614689+0000","last_undegraded":"2024-09-16T11:09:10.614689+0000","last_fullsized":"2024-09-16T11:09:10.614689+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'55","last_scrub_stamp":"2024-09-16T11:09:10.614573+0000","last_deep_scrub":"34'55","last_deep_scrub_stamp":"2024-09-16T11:09:10.614573+0000","last_clean_scrub_stamp":"2024-09-16T11:09:10.614573+0000","objects_scrubbed":0,"log_size":55,"log_dups_size":0,"ondisk_log_size":55,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:29:56.801023+0000","scrub_duration":0.001406416,"objects_trimmed":0,"snaptrim_duration":0.001186241,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":234,"num_read_kb":2061,"num_write":49,"num_write_kb":758,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"29'29","reported_seq":245,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-16T11:09:18.785549+0000","last_change":"2024-09-16T11:09:18.785526+0000","last_active":"2024-09-16T11:09:18.785549+0000","last_peered":"2024-09-16T11:09:18.785549+0000","last_clean":"2024-09-16T11:09:18.785549+0000","last_became_active":"2024-09-16T11:08:07.859480+0000","last_became_peered":"2024-09-16T11:08:07.859480+0000","last_unstale":"2024-09-16T11:09:18.785549+0000","last_undegraded":"2024-09-16T11:09:18.785549+0000","last_fullsized":"2024-09-16T11:09:18.785549+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'29","last_scrub_stamp":"2024-09-16T11:09:18.785446+0000","last_deep_scrub":"29'29","last_deep_scrub_stamp":"2024-09-16T11:09:18.785446+0000","last_clean_scrub_stamp":"2024-09-16T11:09:18.785446+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:49:26.458203+0000","scrub_duration":0.001225851,"objects_trimmed":0,"snaptrim_duration":0.001035411,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":155,"num_read_kb":866,"num_write":30,"num_write_kb":312,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"33'33","reported_seq":175,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-16T11:09:02.822010+0000","last_change":"2024-09-16T11:08:53.882549+0000","last_active":"2024-09-16T11:09:02.822010+0000","last_peered":"2024-09-16T11:09:02.822010+0000","last_clean":"2024-09-16T11:09:02.822010+0000","last_became_active":"2024-09-16T11:08:07.855591+0000","last_became_peered":"2024-09-16T11:08:07.855591+0000","last_unstale":"2024-09-16T11:09:02.822010+0000","last_undegraded":"2024-09-16T11:09:02.822010+0000","last_fullsized":"2024-09-16T11:09:02.822010+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:08:06.827173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:08:06.827173+0000","last_clean_scrub_stamp":"2024-09-16T11:08:06.827173+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":0,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00045890699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":83,"num_read_kb":512,"num_write":38,"num_write_kb":467,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.11","version":"33'88","reported_seq":972,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-16T11:09:09.612356+0000","last_change":"2024-09-16T11:09:09.612308+0000","last_active":"2024-09-16T11:09:09.612356+0000","last_peered":"2024-09-16T11:09:09.612356+0000","last_clean":"2024-09-16T11:09:09.612356+0000","last_became_active":"2024-09-16T11:08:07.856626+0000","last_became_peered":"2024-09-16T11:08:07.856626+0000","last_unstale":"2024-09-16T11:09:09.612356+0000","last_undegraded":"2024-09-16T11:09:09.612356+0000","last_fullsized":"2024-09-16T11:09:09.612356+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'88","last_scrub_stamp":"2024-09-16T11:09:09.612205+0000","last_deep_scrub":"33'88","last_deep_scrub_stamp":"2024-09-16T11:09:09.612205+0000","last_clean_scrub_stamp":"2024-09-16T11:09:09.612205+0000","objects_scrubbed":0,"log_size":88,"log_dups_size":0,"ondisk_log_size":88,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:12:04.711301+0000","scrub_duration":0.0014779610000000001,"objects_trimmed":0,"snaptrim_duration":0.001031704,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1527,"num_read_kb":3994,"num_write":85,"num_write_kb":292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"34'190","reported_seq":382,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-16T11:09:03.793526+0000","last_change":"2024-09-16T11:08:53.898605+0000","last_active":"2024-09-16T11:09:03.793526+0000","last_peered":"2024-09-16T11:09:03.793526+0000","last_clean":"2024-09-16T11:09:03.793526+0000","last_became_active":"2024-09-16T11:08:07.857698+0000","last_became_peered":"2024-09-16T11:08:07.857698+0000","last_unstale":"2024-09-16T11:09:03.793526+0000","last_undegraded":"2024-09-16T11:09:03.793526+0000","last_fullsized":"2024-09-16T11:09:03.793526+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:08:06.827173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:08:06.827173+0000","last_clean_scrub_stamp":"2024-09-16T11:08:06.827173+0000","objects_scrubbed":0,"log_size":190,"log_dups_size":0,"ondisk_log_size":190,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00040617399999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":912,"num_read_kb":991,"num_write":288,"num_write_kb":407,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"34'232","reported_seq":697,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-16T11:09:02.822741+0000","last_change":"2024-09-16T11:08:53.883037+0000","last_active":"2024-09-16T11:09:02.822741+0000","last_peered":"2024-09-16T11:09:02.822741+0000","last_clean":"2024-09-16T11:09:02.822741+0000","last_became_active":"2024-09-16T11:08:07.853749+0000","last_became_peered":"2024-09-16T11:08:07.853749+0000","last_unstale":"2024-09-16T11:09:02.822741+0000","last_undegraded":"2024-09-16T11:09:02.822741+0000","last_fullsized":"2024-09-16T11:09:02.822741+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:08:06.827173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:08:06.827173+0000","last_clean_scrub_stamp":"2024-09-16T11:08:06.827173+0000","objects_scrubbed":0,"log_size":232,"log_dups_size":0,"ondisk_log_size":232,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00063776800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1659,"num_read_kb":2488,"num_write":313,"num_write_kb":1774,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":16005,"num_read_kb":49038,"num_write":3041,"num_write_kb":27918,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2483,"ondisk_log_size":2483,"up":64,"acting":64,"num_store_stats":3},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5,"ondisk_log_size":5,"up":16,"acting":16,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":56,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":54,"ondisk_log_size":54,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359738395,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301816,"kb_used_data":304,"kb_used_omap":1,"kb_used_meta":301502,"kb_avail":94070024,"statfs":{"total":96636764160,"available":96327704576,"internally_reserved":0,"allocated":311296,"data_stored":120927,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308738392},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 11:08:40 2024","interfaces":[{"interface":"back","average":{"1min":0.221,"5min":0.221,"15min":0.221},"min":{"1min":0.14599999999999999,"5min":0.14599999999999999,"15min":0.14599999999999999},"max":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"last":0.23899999999999999},{"interface":"front","average":{"1min":0.23200000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"min":{"1min":0.17000000000000001,"5min":0.17000000000000001,"15min":0.17000000000000001},"max":{"1min":0.31,"5min":0.31,"15min":0.31},"last":0.251}]},{"osd":1,"last update":"Mon Sep 16 11:08:40 2024","interfaces":[{"interface":"back","average":{"1min":0.23799999999999999,"5min":0.23799999999999999,"15min":0.23799999999999999},"min":{"1min":0.159,"5min":0.159,"15min":0.159},"max":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"last":0.22500000000000001},{"interface":"front","average":{"1min":0.252,"5min":0.252,"15min":0.252},"min":{"1min":0.184,"5min":0.184,"15min":0.184},"max":{"1min":0.376,"5min":0.376,"15min":0.376},"last":0.26900000000000002}]}]},{"osd":1,"up_from":8,"seq":34359738396,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":306564,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":305662,"kb_avail":94065276,"statfs":{"total":96636764160,"available":96322842624,"internally_reserved":0,"allocated":913408,"data_stored":712075,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":312998232},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 11:08:37 2024","interfaces":[{"interface":"back","average":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"min":{"1min":0.17399999999999999,"5min":0.17399999999999999,"15min":0.17399999999999999},"max":{"1min":0.376,"5min":0.376,"15min":0.376},"last":0.374},{"interface":"front","average":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"min":{"1min":0.17599999999999999,"5min":0.17599999999999999,"15min":0.17599999999999999},"max":{"1min":0.376,"5min":0.376,"15min":0.376},"last":0.40400000000000003}]},{"osd":2,"last update":"Mon Sep 16 11:08:37 2024","interfaces":[{"interface":"back","average":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"min":{"1min":0.16600000000000001,"5min":0.16600000000000001,"15min":0.16600000000000001},"max":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"last":0.38800000000000001},{"interface":"front","average":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"min":{"1min":0.191,"5min":0.191,"15min":0.191},"max":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"last":0.42499999999999999}]}]},{"osd":0,"up_from":8,"seq":34359738395,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":306556,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":305662,"kb_avail":94065284,"statfs":{"total":96636764160,"available":96322850816,"internally_reserved":0,"allocated":905216,"data_stored":711295,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":312998232},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Sep 16 11:08:40 2024","interfaces":[{"interface":"back","average":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"min":{"1min":0.14599999999999999,"5min":0.14599999999999999,"15min":0.14599999999999999},"max":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"last":0.21099999999999999},{"interface":"front","average":{"1min":0.31,"5min":0.31,"15min":0.31},"min":{"1min":0.16600000000000001,"5min":0.16600000000000001,"15min":0.16600000000000001},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.24399999999999999}]},{"osd":2,"last update":"Mon Sep 16 11:08:40 2024","interfaces":[{"interface":"back","average":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"min":{"1min":0.20300000000000001,"5min":0.20300000000000001,"15min":0.20300000000000001},"max":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"last":0.32900000000000001},{"interface":"front","average":{"1min":0.309,"5min":0.309,"15min":0.309},"min":{"1min":0.186,"5min":0.186,"15min":0.186},"max":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"last":0.23699999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T11:09:38.966 INFO:tasks.ceph:pgid 3.1f last_scrub_stamp 2024-09-16T11:08:06.827173+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=8, tm_sec=6, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=8, tm_sec=55, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T11:09:38.966 INFO:tasks.ceph:pgid 3.1c last_scrub_stamp 2024-09-16T11:08:06.827173+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=8, tm_sec=6, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=8, tm_sec=55, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T11:09:38.966 INFO:tasks.ceph:pgid 3.1b last_scrub_stamp 2024-09-16T11:08:06.827173+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=8, tm_sec=6, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=8, tm_sec=55, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T11:09:38.966 INFO:tasks.ceph:pgid 3.1a last_scrub_stamp 2024-09-16T11:08:06.827173+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=8, tm_sec=6, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=8, tm_sec=55, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T11:09:38.966 INFO:tasks.ceph:pgid 3.19 last_scrub_stamp 2024-09-16T11:08:06.827173+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=8, tm_sec=6, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=8, tm_sec=55, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T11:09:38.967 INFO:tasks.ceph:pgid 3.18 last_scrub_stamp 2024-09-16T11:08:06.827173+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=8, tm_sec=6, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=8, tm_sec=55, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T11:09:38.967 INFO:tasks.ceph:pgid 3.14 last_scrub_stamp 2024-09-16T11:08:06.827173+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=8, tm_sec=6, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=8, tm_sec=55, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T11:09:38.967 INFO:tasks.ceph:pgid 3.0 last_scrub_stamp 2024-09-16T11:08:06.827173+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=8, tm_sec=6, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=8, tm_sec=55, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T11:09:38.967 INFO:tasks.ceph:pgid 3.8 last_scrub_stamp 2024-09-16T11:08:06.827173+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=8, tm_sec=6, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=8, tm_sec=55, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T11:09:38.967 INFO:tasks.ceph:pgid 3.9 last_scrub_stamp 2024-09-16T11:08:06.827173+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=8, tm_sec=6, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=8, tm_sec=55, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T11:09:38.967 INFO:tasks.ceph:pgid 3.10 last_scrub_stamp 2024-09-16T11:08:06.827173+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=8, tm_sec=6, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=8, tm_sec=55, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T11:09:38.968 INFO:tasks.ceph:pgid 3.12 last_scrub_stamp 2024-09-16T11:08:06.827173+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=8, tm_sec=6, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=8, tm_sec=55, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T11:09:38.968 INFO:tasks.ceph:pgid 3.13 last_scrub_stamp 2024-09-16T11:08:06.827173+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=8, tm_sec=6, tm_wday=0, tm_yday=260, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=16, tm_hour=11, tm_min=8, tm_sec=55, tm_wday=0, tm_yday=260, tm_isdst=0) 2024-09-16T11:09:38.968 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-09-16T11:09:58.969 DEBUG:teuthology.orchestra.run.smithi102:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T11:09:59.251 INFO:teuthology.orchestra.run.smithi102.stdout: 2024-09-16T11:09:59.252 INFO:teuthology.orchestra.run.smithi102.stderr:dumped all 2024-09-16T11:09:59.263 INFO:teuthology.orchestra.run.smithi102.stdout:{"pg_ready":true,"pg_map":{"version":105,"stamp":"2024-09-16T11:09:57.672703+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590406,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":16061,"num_read_kb":49084,"num_write":3122,"num_write_kb":29026,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2544,"ondisk_log_size":2544,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":82,"num_osds":3,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":283115520,"kb_used":919096,"kb_used_data":2080,"kb_used_omap":4,"kb_used_meta":916987,"kb_avail":282196424,"statfs":{"total":289910292480,"available":288969138176,"internally_reserved":0,"allocated":2129920,"data_stored":1544297,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":5112,"internal_metadata":938994696},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002372"},"pg_stats":[{"pgid":"3.1f","version":"35'35","reported_seq":228,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-16T11:09:50.687876+0000","last_change":"2024-09-16T11:09:50.687843+0000","last_active":"2024-09-16T11:09:50.687876+0000","last_peered":"2024-09-16T11:09:50.687876+0000","last_clean":"2024-09-16T11:09:50.687876+0000","last_became_active":"2024-09-16T11:08:07.859102+0000","last_became_peered":"2024-09-16T11:08:07.859102+0000","last_unstale":"2024-09-16T11:09:50.687876+0000","last_undegraded":"2024-09-16T11:09:50.687876+0000","last_fullsized":"2024-09-16T11:09:50.687876+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"35'35","last_scrub_stamp":"2024-09-16T11:09:50.687754+0000","last_deep_scrub":"35'35","last_deep_scrub_stamp":"2024-09-16T11:09:50.687754+0000","last_clean_scrub_stamp":"2024-09-16T11:09:50.687754+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:38:03.522207+0000","scrub_duration":0.0013538560000000001,"objects_trimmed":0,"snaptrim_duration":0.00095524,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":130,"num_read_kb":1007,"num_write":36,"num_write_kb":861,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"35'187","reported_seq":868,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-16T11:09:24.625883+0000","last_change":"2024-09-16T11:09:24.625856+0000","last_active":"2024-09-16T11:09:24.625883+0000","last_peered":"2024-09-16T11:09:24.625883+0000","last_clean":"2024-09-16T11:09:24.625883+0000","last_became_active":"2024-09-16T11:08:07.856092+0000","last_became_peered":"2024-09-16T11:08:07.856092+0000","last_unstale":"2024-09-16T11:09:24.625883+0000","last_undegraded":"2024-09-16T11:09:24.625883+0000","last_fullsized":"2024-09-16T11:09:24.625883+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"35'187","last_scrub_stamp":"2024-09-16T11:09:24.625766+0000","last_deep_scrub":"35'187","last_deep_scrub_stamp":"2024-09-16T11:09:24.625766+0000","last_clean_scrub_stamp":"2024-09-16T11:09:24.625766+0000","objects_scrubbed":0,"log_size":187,"log_dups_size":0,"ondisk_log_size":187,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:17:47.522929+0000","scrub_duration":0.0013864529999999999,"objects_trimmed":0,"snaptrim_duration":0.001038806,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1989,"num_read_kb":2691,"num_write":225,"num_write_kb":434,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1d","version":"35'287","reported_seq":373,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-16T11:09:26.581258+0000","last_change":"2024-09-16T11:09:26.581231+0000","last_active":"2024-09-16T11:09:26.581258+0000","last_peered":"2024-09-16T11:09:26.581258+0000","last_clean":"2024-09-16T11:09:26.581258+0000","last_became_active":"2024-09-16T11:08:07.847513+0000","last_became_peered":"2024-09-16T11:08:07.847513+0000","last_unstale":"2024-09-16T11:09:26.581258+0000","last_undegraded":"2024-09-16T11:09:26.581258+0000","last_fullsized":"2024-09-16T11:09:26.581258+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"35'287","last_scrub_stamp":"2024-09-16T11:09:26.581133+0000","last_deep_scrub":"35'287","last_deep_scrub_stamp":"2024-09-16T11:09:26.581133+0000","last_clean_scrub_stamp":"2024-09-16T11:09:26.581133+0000","objects_scrubbed":0,"log_size":287,"log_dups_size":0,"ondisk_log_size":287,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:10:04.869239+0000","scrub_duration":0.001488461,"objects_trimmed":0,"snaptrim_duration":0.000667575,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1265,"num_read_kb":1036,"num_write":459,"num_write_kb":602,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"37'37","reported_seq":248,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-16T11:09:49.759084+0000","last_change":"2024-09-16T11:09:49.759053+0000","last_active":"2024-09-16T11:09:49.759084+0000","last_peered":"2024-09-16T11:09:49.759084+0000","last_clean":"2024-09-16T11:09:49.759084+0000","last_became_active":"2024-09-16T11:08:07.855430+0000","last_became_peered":"2024-09-16T11:08:07.855430+0000","last_unstale":"2024-09-16T11:09:49.759084+0000","last_undegraded":"2024-09-16T11:09:49.759084+0000","last_fullsized":"2024-09-16T11:09:49.759084+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'37","last_scrub_stamp":"2024-09-16T11:09:49.758945+0000","last_deep_scrub":"37'37","last_deep_scrub_stamp":"2024-09-16T11:09:49.758945+0000","last_clean_scrub_stamp":"2024-09-16T11:09:49.758945+0000","objects_scrubbed":1,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:10:07.131144+0000","scrub_duration":0.0027211169999999999,"objects_trimmed":0,"snaptrim_duration":0.00097256700000000003,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":177,"num_read_kb":260,"num_write":47,"num_write_kb":566,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"34'165","reported_seq":869,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-16T11:09:36.657249+0000","last_change":"2024-09-16T11:09:36.657218+0000","last_active":"2024-09-16T11:09:36.657249+0000","last_peered":"2024-09-16T11:09:36.657249+0000","last_clean":"2024-09-16T11:09:36.657249+0000","last_became_active":"2024-09-16T11:08:07.856827+0000","last_became_peered":"2024-09-16T11:08:07.856827+0000","last_unstale":"2024-09-16T11:09:36.657249+0000","last_undegraded":"2024-09-16T11:09:36.657249+0000","last_fullsized":"2024-09-16T11:09:36.657249+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'165","last_scrub_stamp":"2024-09-16T11:09:36.657131+0000","last_deep_scrub":"34'165","last_deep_scrub_stamp":"2024-09-16T11:09:36.657131+0000","last_clean_scrub_stamp":"2024-09-16T11:09:36.657131+0000","objects_scrubbed":0,"log_size":165,"log_dups_size":0,"ondisk_log_size":165,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:10.911657+0000","scrub_duration":0.001438094,"objects_trimmed":0,"snaptrim_duration":0.00081398199999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1407,"num_read_kb":2244,"num_write":140,"num_write_kb":887,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1a","version":"37'14","reported_seq":410,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-16T11:09:41.753247+0000","last_change":"2024-09-16T11:09:41.753213+0000","last_active":"2024-09-16T11:09:41.753247+0000","last_peered":"2024-09-16T11:09:41.753247+0000","last_clean":"2024-09-16T11:09:41.753247+0000","last_became_active":"2024-09-16T11:08:07.853372+0000","last_became_peered":"2024-09-16T11:08:07.853372+0000","last_unstale":"2024-09-16T11:09:41.753247+0000","last_undegraded":"2024-09-16T11:09:41.753247+0000","last_fullsized":"2024-09-16T11:09:41.753247+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'14","last_scrub_stamp":"2024-09-16T11:09:41.753091+0000","last_deep_scrub":"37'14","last_deep_scrub_stamp":"2024-09-16T11:09:41.753091+0000","last_clean_scrub_stamp":"2024-09-16T11:09:41.753091+0000","objects_scrubbed":1,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:54:35.102000+0000","scrub_duration":0.0027190890000000001,"objects_trimmed":0,"snaptrim_duration":0.00082456000000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":325,"num_read_kb":343,"num_write":13,"num_write_kb":81,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.19","version":"35'47","reported_seq":312,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-16T11:09:44.725617+0000","last_change":"2024-09-16T11:09:44.725579+0000","last_active":"2024-09-16T11:09:44.725617+0000","last_peered":"2024-09-16T11:09:44.725617+0000","last_clean":"2024-09-16T11:09:44.725617+0000","last_became_active":"2024-09-16T11:08:07.852874+0000","last_became_peered":"2024-09-16T11:08:07.852874+0000","last_unstale":"2024-09-16T11:09:44.725617+0000","last_undegraded":"2024-09-16T11:09:44.725617+0000","last_fullsized":"2024-09-16T11:09:44.725617+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"35'47","last_scrub_stamp":"2024-09-16T11:09:44.725488+0000","last_deep_scrub":"35'47","last_deep_scrub_stamp":"2024-09-16T11:09:44.725488+0000","last_clean_scrub_stamp":"2024-09-16T11:09:44.725488+0000","objects_scrubbed":0,"log_size":47,"log_dups_size":0,"ondisk_log_size":47,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:37:57.559942+0000","scrub_duration":0.0014811819999999999,"objects_trimmed":0,"snaptrim_duration":0.00052581399999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":214,"num_read_kb":667,"num_write":46,"num_write_kb":746,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.18","version":"35'18","reported_seq":92,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-16T11:09:36.528103+0000","last_change":"2024-09-16T11:09:36.528074+0000","last_active":"2024-09-16T11:09:36.528103+0000","last_peered":"2024-09-16T11:09:36.528103+0000","last_clean":"2024-09-16T11:09:36.528103+0000","last_became_active":"2024-09-16T11:08:07.848434+0000","last_became_peered":"2024-09-16T11:08:07.848434+0000","last_unstale":"2024-09-16T11:09:36.528103+0000","last_undegraded":"2024-09-16T11:09:36.528103+0000","last_fullsized":"2024-09-16T11:09:36.528103+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"35'18","last_scrub_stamp":"2024-09-16T11:09:36.527943+0000","last_deep_scrub":"35'18","last_deep_scrub_stamp":"2024-09-16T11:09:36.527943+0000","last_clean_scrub_stamp":"2024-09-16T11:09:36.527943+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:18:30.704674+0000","scrub_duration":0.001527002,"objects_trimmed":0,"snaptrim_duration":0.00043492900000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":18,"num_write_kb":60,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.17","version":"31'25","reported_seq":124,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-16T11:09:23.766121+0000","last_change":"2024-09-16T11:09:23.766090+0000","last_active":"2024-09-16T11:09:23.766121+0000","last_peered":"2024-09-16T11:09:23.766121+0000","last_clean":"2024-09-16T11:09:23.766121+0000","last_became_active":"2024-09-16T11:08:07.859097+0000","last_became_peered":"2024-09-16T11:08:07.859097+0000","last_unstale":"2024-09-16T11:09:23.766121+0000","last_undegraded":"2024-09-16T11:09:23.766121+0000","last_fullsized":"2024-09-16T11:09:23.766121+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"31'25","last_scrub_stamp":"2024-09-16T11:09:23.765992+0000","last_deep_scrub":"31'25","last_deep_scrub_stamp":"2024-09-16T11:09:23.765992+0000","last_clean_scrub_stamp":"2024-09-16T11:09:23.765992+0000","objects_scrubbed":0,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:10:02.054098+0000","scrub_duration":0.0014903340000000001,"objects_trimmed":0,"snaptrim_duration":0.001161507,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":30,"num_read_kb":66,"num_write":24,"num_write_kb":325,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"32'25","reported_seq":153,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-16T11:09:18.566133+0000","last_change":"2024-09-16T11:09:18.566112+0000","last_active":"2024-09-16T11:09:18.566133+0000","last_peered":"2024-09-16T11:09:18.566133+0000","last_clean":"2024-09-16T11:09:18.566133+0000","last_became_active":"2024-09-16T11:08:07.856007+0000","last_became_peered":"2024-09-16T11:08:07.856007+0000","last_unstale":"2024-09-16T11:09:18.566133+0000","last_undegraded":"2024-09-16T11:09:18.566133+0000","last_fullsized":"2024-09-16T11:09:18.566133+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'25","last_scrub_stamp":"2024-09-16T11:09:18.566035+0000","last_deep_scrub":"32'25","last_deep_scrub_stamp":"2024-09-16T11:09:18.566035+0000","last_clean_scrub_stamp":"2024-09-16T11:09:18.566035+0000","objects_scrubbed":0,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:07:14.883915+0000","scrub_duration":0.0011610380000000001,"objects_trimmed":0,"snaptrim_duration":0.0011111560000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":66,"num_read_kb":1713,"num_write":38,"num_write_kb":1539,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"33'31","reported_seq":185,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-16T11:09:32.644690+0000","last_change":"2024-09-16T11:09:32.644664+0000","last_active":"2024-09-16T11:09:32.644690+0000","last_peered":"2024-09-16T11:09:32.644690+0000","last_clean":"2024-09-16T11:09:32.644690+0000","last_became_active":"2024-09-16T11:08:07.858375+0000","last_became_peered":"2024-09-16T11:08:07.858375+0000","last_unstale":"2024-09-16T11:09:32.644690+0000","last_undegraded":"2024-09-16T11:09:32.644690+0000","last_fullsized":"2024-09-16T11:09:32.644690+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'31","last_scrub_stamp":"2024-09-16T11:09:32.644569+0000","last_deep_scrub":"33'31","last_deep_scrub_stamp":"2024-09-16T11:09:32.644569+0000","last_clean_scrub_stamp":"2024-09-16T11:09:32.644569+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:36:18.603929+0000","scrub_duration":0.001451052,"objects_trimmed":0,"snaptrim_duration":0.00079626899999999995,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":92,"num_read_kb":770,"num_write":36,"num_write_kb":1093,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.14","version":"35'141","reported_seq":318,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-16T11:09:40.797470+0000","last_change":"2024-09-16T11:09:40.797440+0000","last_active":"2024-09-16T11:09:40.797470+0000","last_peered":"2024-09-16T11:09:40.797470+0000","last_clean":"2024-09-16T11:09:40.797470+0000","last_became_active":"2024-09-16T11:08:07.853189+0000","last_became_peered":"2024-09-16T11:08:07.853189+0000","last_unstale":"2024-09-16T11:09:40.797470+0000","last_undegraded":"2024-09-16T11:09:40.797470+0000","last_fullsized":"2024-09-16T11:09:40.797470+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"35'141","last_scrub_stamp":"2024-09-16T11:09:40.797326+0000","last_deep_scrub":"35'141","last_deep_scrub_stamp":"2024-09-16T11:09:40.797326+0000","last_clean_scrub_stamp":"2024-09-16T11:09:40.797326+0000","objects_scrubbed":0,"log_size":141,"log_dups_size":0,"ondisk_log_size":141,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:12:02.642865+0000","scrub_duration":0.0014123810000000001,"objects_trimmed":0,"snaptrim_duration":0.00087565099999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":523,"num_read_kb":930,"num_write":189,"num_write_kb":943,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":48,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-16T11:09:03.797856+0000","last_change":"2024-09-16T11:09:03.797818+0000","last_active":"2024-09-16T11:09:03.797856+0000","last_peered":"2024-09-16T11:09:03.797856+0000","last_clean":"2024-09-16T11:09:03.797856+0000","last_became_active":"2024-09-16T11:07:44.446646+0000","last_became_peered":"2024-09-16T11:07:44.446646+0000","last_unstale":"2024-09-16T11:09:03.797856+0000","last_undegraded":"2024-09-16T11:09:03.797856+0000","last_fullsized":"2024-09-16T11:09:03.797856+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:09:03.797714+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:09:03.797714+0000","last_clean_scrub_stamp":"2024-09-16T11:09:03.797714+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:29:49.984164+0000","scrub_duration":0.0062555400000000004,"objects_trimmed":0,"snaptrim_duration":0.000337344,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.6","version":"35'83","reported_seq":163,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-16T11:09:21.742649+0000","last_change":"2024-09-16T11:09:21.742613+0000","last_active":"2024-09-16T11:09:21.742649+0000","last_peered":"2024-09-16T11:09:21.742649+0000","last_clean":"2024-09-16T11:09:21.742649+0000","last_became_active":"2024-09-16T11:08:07.858208+0000","last_became_peered":"2024-09-16T11:08:07.858208+0000","last_unstale":"2024-09-16T11:09:21.742649+0000","last_undegraded":"2024-09-16T11:09:21.742649+0000","last_fullsized":"2024-09-16T11:09:21.742649+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"35'83","last_scrub_stamp":"2024-09-16T11:09:21.742515+0000","last_deep_scrub":"35'83","last_deep_scrub_stamp":"2024-09-16T11:09:21.742515+0000","last_clean_scrub_stamp":"2024-09-16T11:09:21.742515+0000","objects_scrubbed":0,"log_size":83,"log_dups_size":0,"ondisk_log_size":83,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:17:44.639678+0000","scrub_duration":0.0015147889999999999,"objects_trimmed":0,"snaptrim_duration":0.00098881699999999991,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":259,"num_read_kb":213,"num_write":114,"num_write_kb":675,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":48,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-16T11:09:07.770754+0000","last_change":"2024-09-16T11:09:07.770730+0000","last_active":"2024-09-16T11:09:07.770754+0000","last_peered":"2024-09-16T11:09:07.770754+0000","last_clean":"2024-09-16T11:09:07.770754+0000","last_became_active":"2024-09-16T11:07:44.446406+0000","last_became_peered":"2024-09-16T11:07:44.446406+0000","last_unstale":"2024-09-16T11:09:07.770754+0000","last_undegraded":"2024-09-16T11:09:07.770754+0000","last_fullsized":"2024-09-16T11:09:07.770754+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:09:07.770642+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:09:07.770642+0000","last_clean_scrub_stamp":"2024-09-16T11:09:07.770642+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:29:11.031172+0000","scrub_duration":0.001319305,"objects_trimmed":0,"snaptrim_duration":0.00062301399999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"35'44","reported_seq":342,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-16T11:09:21.594392+0000","last_change":"2024-09-16T11:09:21.594367+0000","last_active":"2024-09-16T11:09:21.594392+0000","last_peered":"2024-09-16T11:09:21.594392+0000","last_clean":"2024-09-16T11:09:21.594392+0000","last_became_active":"2024-09-16T11:08:07.856393+0000","last_became_peered":"2024-09-16T11:08:07.856393+0000","last_unstale":"2024-09-16T11:09:21.594392+0000","last_undegraded":"2024-09-16T11:09:21.594392+0000","last_fullsized":"2024-09-16T11:09:21.594392+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"35'44","last_scrub_stamp":"2024-09-16T11:09:21.594260+0000","last_deep_scrub":"35'44","last_deep_scrub_stamp":"2024-09-16T11:09:21.594260+0000","last_clean_scrub_stamp":"2024-09-16T11:09:21.594260+0000","objects_scrubbed":0,"log_size":44,"log_dups_size":0,"ondisk_log_size":44,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:49:29.267016+0000","scrub_duration":0.0013851639999999999,"objects_trimmed":0,"snaptrim_duration":0.00064218800000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":241,"num_read_kb":2845,"num_write":55,"num_write_kb":2044,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":48,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-16T11:09:17.757451+0000","last_change":"2024-09-16T11:09:17.757404+0000","last_active":"2024-09-16T11:09:17.757451+0000","last_peered":"2024-09-16T11:09:17.757451+0000","last_clean":"2024-09-16T11:09:17.757451+0000","last_became_active":"2024-09-16T11:07:44.446498+0000","last_became_peered":"2024-09-16T11:07:44.446498+0000","last_unstale":"2024-09-16T11:09:17.757451+0000","last_undegraded":"2024-09-16T11:09:17.757451+0000","last_fullsized":"2024-09-16T11:09:17.757451+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:09:17.757291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:09:17.757291+0000","last_clean_scrub_stamp":"2024-09-16T11:09:17.757291+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T22:17:40.654454+0000","scrub_duration":0.001523022,"objects_trimmed":0,"snaptrim_duration":0.00084251400000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"32'67","reported_seq":291,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-16T11:09:19.738684+0000","last_change":"2024-09-16T11:09:19.738637+0000","last_active":"2024-09-16T11:09:19.738684+0000","last_peered":"2024-09-16T11:09:19.738684+0000","last_clean":"2024-09-16T11:09:19.738684+0000","last_became_active":"2024-09-16T11:08:07.853470+0000","last_became_peered":"2024-09-16T11:08:07.853470+0000","last_unstale":"2024-09-16T11:09:19.738684+0000","last_undegraded":"2024-09-16T11:09:19.738684+0000","last_fullsized":"2024-09-16T11:09:19.738684+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"32'67","last_scrub_stamp":"2024-09-16T11:09:19.738524+0000","last_deep_scrub":"32'67","last_deep_scrub_stamp":"2024-09-16T11:09:19.738524+0000","last_clean_scrub_stamp":"2024-09-16T11:09:19.738524+0000","objects_scrubbed":0,"log_size":67,"log_dups_size":0,"ondisk_log_size":67,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T16:09:58.026631+0000","scrub_duration":0.0016065560000000001,"objects_trimmed":0,"snaptrim_duration":0.001033672,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":360,"num_read_kb":1468,"num_write":100,"num_write_kb":888,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":48,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-16T11:09:14.743581+0000","last_change":"2024-09-16T11:09:14.743549+0000","last_active":"2024-09-16T11:09:14.743581+0000","last_peered":"2024-09-16T11:09:14.743581+0000","last_clean":"2024-09-16T11:09:14.743581+0000","last_became_active":"2024-09-16T11:07:44.446413+0000","last_became_peered":"2024-09-16T11:07:44.446413+0000","last_unstale":"2024-09-16T11:09:14.743581+0000","last_undegraded":"2024-09-16T11:09:14.743581+0000","last_fullsized":"2024-09-16T11:09:14.743581+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:09:14.743448+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:09:14.743448+0000","last_clean_scrub_stamp":"2024-09-16T11:09:14.743448+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:49:22.416205+0000","scrub_duration":0.0014080869999999999,"objects_trimmed":0,"snaptrim_duration":0.00059042800000000005,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"34'78","reported_seq":502,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-16T11:09:14.635358+0000","last_change":"2024-09-16T11:09:14.635326+0000","last_active":"2024-09-16T11:09:14.635358+0000","last_peered":"2024-09-16T11:09:14.635358+0000","last_clean":"2024-09-16T11:09:14.635358+0000","last_became_active":"2024-09-16T11:08:07.855280+0000","last_became_peered":"2024-09-16T11:08:07.855280+0000","last_unstale":"2024-09-16T11:09:14.635358+0000","last_undegraded":"2024-09-16T11:09:14.635358+0000","last_fullsized":"2024-09-16T11:09:14.635358+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'78","last_scrub_stamp":"2024-09-16T11:09:14.635225+0000","last_deep_scrub":"34'78","last_deep_scrub_stamp":"2024-09-16T11:09:14.635225+0000","last_clean_scrub_stamp":"2024-09-16T11:09:14.635225+0000","objects_scrubbed":0,"log_size":78,"log_dups_size":0,"ondisk_log_size":78,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:29:17.895755+0000","scrub_duration":0.0015130829999999999,"objects_trimmed":0,"snaptrim_duration":0.00040932799999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":396,"num_read_kb":2349,"num_write":79,"num_write_kb":1069,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3","version":"35'86","reported_seq":288,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-16T11:09:07.764657+0000","last_change":"2024-09-16T11:09:07.764626+0000","last_active":"2024-09-16T11:09:07.764657+0000","last_peered":"2024-09-16T11:09:07.764657+0000","last_clean":"2024-09-16T11:09:07.764657+0000","last_became_active":"2024-09-16T11:08:07.847550+0000","last_became_peered":"2024-09-16T11:08:07.847550+0000","last_unstale":"2024-09-16T11:09:07.764657+0000","last_undegraded":"2024-09-16T11:09:07.764657+0000","last_fullsized":"2024-09-16T11:09:07.764657+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"35'86","last_scrub_stamp":"2024-09-16T11:09:07.764522+0000","last_deep_scrub":"35'86","last_deep_scrub_stamp":"2024-09-16T11:09:07.764522+0000","last_clean_scrub_stamp":"2024-09-16T11:09:07.764522+0000","objects_scrubbed":1,"log_size":86,"log_dups_size":0,"ondisk_log_size":86,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:29:53.950973+0000","scrub_duration":0.0017137980000000001,"objects_trimmed":0,"snaptrim_duration":0.000875419,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":164,"num_read_kb":968,"num_write":116,"num_write_kb":2579,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"35'3","reported_seq":50,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-16T11:08:58.393319+0000","last_change":"2024-09-16T11:08:57.637442+0000","last_active":"2024-09-16T11:08:58.393319+0000","last_peered":"2024-09-16T11:08:58.393319+0000","last_clean":"2024-09-16T11:08:58.393319+0000","last_became_active":"2024-09-16T11:07:44.449069+0000","last_became_peered":"2024-09-16T11:07:44.449069+0000","last_unstale":"2024-09-16T11:08:58.393319+0000","last_undegraded":"2024-09-16T11:08:58.393319+0000","last_fullsized":"2024-09-16T11:08:58.393319+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"35'3","last_scrub_stamp":"2024-09-16T11:08:57.637376+0000","last_deep_scrub":"35'3","last_deep_scrub_stamp":"2024-09-16T11:08:57.637376+0000","last_clean_scrub_stamp":"2024-09-16T11:08:57.637376+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:59:04.639271+0000","scrub_duration":0.0023648480000000001,"objects_trimmed":0,"snaptrim_duration":0.00029305100000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.0","version":"34'47","reported_seq":230,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-16T11:09:32.681208+0000","last_change":"2024-09-16T11:09:32.681175+0000","last_active":"2024-09-16T11:09:32.681208+0000","last_peered":"2024-09-16T11:09:32.681208+0000","last_clean":"2024-09-16T11:09:32.681208+0000","last_became_active":"2024-09-16T11:08:07.855258+0000","last_became_peered":"2024-09-16T11:08:07.855258+0000","last_unstale":"2024-09-16T11:09:32.681208+0000","last_undegraded":"2024-09-16T11:09:32.681208+0000","last_fullsized":"2024-09-16T11:09:32.681208+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'47","last_scrub_stamp":"2024-09-16T11:09:32.681083+0000","last_deep_scrub":"34'47","last_deep_scrub_stamp":"2024-09-16T11:09:32.681083+0000","last_clean_scrub_stamp":"2024-09-16T11:09:32.681083+0000","objects_scrubbed":0,"log_size":47,"log_dups_size":0,"ondisk_log_size":47,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T15:21:06.935609+0000","scrub_duration":0.0013339700000000001,"objects_trimmed":0,"snaptrim_duration":0.00075381500000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":141,"num_read_kb":1673,"num_write":53,"num_write_kb":1263,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":47,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-16T11:09:06.562694+0000","last_change":"2024-09-16T11:09:06.562665+0000","last_active":"2024-09-16T11:09:06.562694+0000","last_peered":"2024-09-16T11:09:06.562694+0000","last_clean":"2024-09-16T11:09:06.562694+0000","last_became_active":"2024-09-16T11:07:44.447533+0000","last_became_peered":"2024-09-16T11:07:44.447533+0000","last_unstale":"2024-09-16T11:09:06.562694+0000","last_undegraded":"2024-09-16T11:09:06.562694+0000","last_fullsized":"2024-09-16T11:09:06.562694+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:09:06.562572+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:09:06.562572+0000","last_clean_scrub_stamp":"2024-09-16T11:09:06.562572+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:43:14.347543+0000","scrub_duration":0.0013831399999999999,"objects_trimmed":0,"snaptrim_duration":0.000230378,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1","version":"33'30","reported_seq":185,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-16T11:09:03.804035+0000","last_change":"2024-09-16T11:09:03.804009+0000","last_active":"2024-09-16T11:09:03.804035+0000","last_peered":"2024-09-16T11:09:03.804035+0000","last_clean":"2024-09-16T11:09:03.804035+0000","last_became_active":"2024-09-16T11:08:07.847955+0000","last_became_peered":"2024-09-16T11:08:07.847955+0000","last_unstale":"2024-09-16T11:09:03.804035+0000","last_undegraded":"2024-09-16T11:09:03.804035+0000","last_fullsized":"2024-09-16T11:09:03.804035+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'30","last_scrub_stamp":"2024-09-16T11:09:03.803902+0000","last_deep_scrub":"33'30","last_deep_scrub_stamp":"2024-09-16T11:09:03.803902+0000","last_clean_scrub_stamp":"2024-09-16T11:09:03.803902+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:43:11.588873+0000","scrub_duration":0.001499385,"objects_trimmed":0,"snaptrim_duration":0.00029393900000000001,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":87,"num_read_kb":1164,"num_write":31,"num_write_kb":740,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":47,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-16T11:09:01.573333+0000","last_change":"2024-09-16T11:09:01.573294+0000","last_active":"2024-09-16T11:09:01.573333+0000","last_peered":"2024-09-16T11:09:01.573333+0000","last_clean":"2024-09-16T11:09:01.573333+0000","last_became_active":"2024-09-16T11:07:44.448153+0000","last_became_peered":"2024-09-16T11:07:44.448153+0000","last_unstale":"2024-09-16T11:09:01.573333+0000","last_undegraded":"2024-09-16T11:09:01.573333+0000","last_fullsized":"2024-09-16T11:09:01.573333+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T11:09:01.573204+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T11:09:01.573204+0000","last_clean_scrub_stamp":"2024-09-16T11:09:01.573204+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:59:08.575099+0000","scrub_duration":0.001347532,"objects_trimmed":0,"snaptrim_duration":0.00024792500000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"34'46","reported_seq":296,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-16T11:09:02.822765+0000","last_change":"2024-09-16T11:08:59.685510+0000","last_active":"2024-09-16T11:09:02.822765+0000","last_peered":"2024-09-16T11:09:02.822765+0000","last_clean":"2024-09-16T11:09:02.822765+0000","last_became_active":"2024-09-16T11:08:07.853307+0000","last_became_peered":"2024-09-16T11:08:07.853307+0000","last_unstale":"2024-09-16T11:09:02.822765+0000","last_undegraded":"2024-09-16T11:09:02.822765+0000","last_fullsized":"2024-09-16T11:09:02.822765+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'46","last_scrub_stamp":"2024-09-16T11:08:59.685380+0000","last_deep_scrub":"34'46","last_deep_scrub_stamp":"2024-09-16T11:08:59.685380+0000","last_clean_scrub_stamp":"2024-09-16T11:08:59.685380+0000","objects_scrubbed":0,"log_size":46,"log_dups_size":0,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:43:07.470351+0000","scrub_duration":0.0015725050000000001,"objects_trimmed":0,"snaptrim_duration":0.0011803549999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":203,"num_read_kb":1989,"num_write":39,"num_write_kb":217,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"37'2","reported_seq":50,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-16T11:09:11.724730+0000","last_change":"2024-09-16T11:09:11.724697+0000","last_active":"2024-09-16T11:09:11.724730+0000","last_peered":"2024-09-16T11:09:11.724730+0000","last_clean":"2024-09-16T11:09:11.724730+0000","last_became_active":"2024-09-16T11:07:44.446367+0000","last_became_peered":"2024-09-16T11:07:44.446367+0000","last_unstale":"2024-09-16T11:09:11.724730+0000","last_undegraded":"2024-09-16T11:09:11.724730+0000","last_fullsized":"2024-09-16T11:09:11.724730+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"37'2","last_scrub_stamp":"2024-09-16T11:09:11.724600+0000","last_deep_scrub":"37'2","last_deep_scrub_stamp":"2024-09-16T11:09:11.724600+0000","last_clean_scrub_stamp":"2024-09-16T11:09:11.724600+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:07:08.042481+0000","scrub_duration":0.0026358029999999999,"objects_trimmed":0,"snaptrim_duration":0.00038771200000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"36'54","reported_seq":115,"reported_epoch":36,"state":"active+clean","last_fresh":"2024-09-16T11:09:02.760758+0000","last_change":"2024-09-16T11:09:02.760678+0000","last_active":"2024-09-16T11:09:02.760758+0000","last_peered":"2024-09-16T11:09:02.760758+0000","last_clean":"2024-09-16T11:09:02.760758+0000","last_became_active":"2024-09-16T11:07:39.420564+0000","last_became_peered":"2024-09-16T11:07:39.420564+0000","last_unstale":"2024-09-16T11:09:02.760758+0000","last_undegraded":"2024-09-16T11:09:02.760758+0000","last_fullsized":"2024-09-16T11:09:02.760758+0000","mapping_epoch":9,"log_start":"0'0","ondisk_log_start":"0'0","created":9,"last_epoch_clean":10,"parent":"0.0","parent_split_bits":0,"last_scrub":"36'54","last_scrub_stamp":"2024-09-16T11:09:02.760577+0000","last_deep_scrub":"36'54","last_deep_scrub_stamp":"2024-09-16T11:09:02.760577+0000","last_clean_scrub_stamp":"2024-09-16T11:09:02.760577+0000","objects_scrubbed":2,"log_size":54,"log_dups_size":0,"ondisk_log_size":54,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:11:57.859673+0000","scrub_duration":0.0048002319999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":56,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"33'41","reported_seq":473,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-16T11:09:35.553377+0000","last_change":"2024-09-16T11:09:35.553352+0000","last_active":"2024-09-16T11:09:35.553377+0000","last_peered":"2024-09-16T11:09:35.553377+0000","last_clean":"2024-09-16T11:09:35.553377+0000","last_became_active":"2024-09-16T11:08:07.857280+0000","last_became_peered":"2024-09-16T11:08:07.857280+0000","last_unstale":"2024-09-16T11:09:35.553377+0000","last_undegraded":"2024-09-16T11:09:35.553377+0000","last_fullsized":"2024-09-16T11:09:35.553377+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'41","last_scrub_stamp":"2024-09-16T11:09:35.553225+0000","last_deep_scrub":"33'41","last_deep_scrub_stamp":"2024-09-16T11:09:35.553225+0000","last_clean_scrub_stamp":"2024-09-16T11:09:35.553225+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:36:21.512585+0000","scrub_duration":0.0016016890000000001,"objects_trimmed":0,"snaptrim_duration":0.001178323,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":381,"num_read_kb":3183,"num_write":40,"num_write_kb":936,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.9","version":"35'18","reported_seq":91,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-16T11:09:33.609348+0000","last_change":"2024-09-16T11:09:33.609309+0000","last_active":"2024-09-16T11:09:33.609348+0000","last_peered":"2024-09-16T11:09:33.609348+0000","last_clean":"2024-09-16T11:09:33.609348+0000","last_became_active":"2024-09-16T11:08:07.850403+0000","last_became_peered":"2024-09-16T11:08:07.850403+0000","last_unstale":"2024-09-16T11:09:33.609348+0000","last_undegraded":"2024-09-16T11:09:33.609348+0000","last_fullsized":"2024-09-16T11:09:33.609348+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"35'18","last_scrub_stamp":"2024-09-16T11:09:33.609194+0000","last_deep_scrub":"35'18","last_deep_scrub_stamp":"2024-09-16T11:09:33.609194+0000","last_clean_scrub_stamp":"2024-09-16T11:09:33.609194+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:18:27.785925+0000","scrub_duration":0.0014861519999999999,"objects_trimmed":0,"snaptrim_duration":0.00025477399999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":55,"num_write":21,"num_write_kb":629,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.a","version":"34'51","reported_seq":389,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-16T11:09:15.758425+0000","last_change":"2024-09-16T11:09:15.758394+0000","last_active":"2024-09-16T11:09:15.758425+0000","last_peered":"2024-09-16T11:09:15.758425+0000","last_clean":"2024-09-16T11:09:15.758425+0000","last_became_active":"2024-09-16T11:08:07.857407+0000","last_became_peered":"2024-09-16T11:08:07.857407+0000","last_unstale":"2024-09-16T11:09:15.758425+0000","last_undegraded":"2024-09-16T11:09:15.758425+0000","last_fullsized":"2024-09-16T11:09:15.758425+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'51","last_scrub_stamp":"2024-09-16T11:09:15.758293+0000","last_deep_scrub":"34'51","last_deep_scrub_stamp":"2024-09-16T11:09:15.758293+0000","last_clean_scrub_stamp":"2024-09-16T11:09:15.758293+0000","objects_scrubbed":1,"log_size":51,"log_dups_size":0,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:07:12.076173+0000","scrub_duration":0.0017331250000000001,"objects_trimmed":0,"snaptrim_duration":0.00055050900000000002,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":289,"num_read_kb":3215,"num_write":55,"num_write_kb":1438,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"34'104","reported_seq":901,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-16T11:09:28.657036+0000","last_change":"2024-09-16T11:09:28.656995+0000","last_active":"2024-09-16T11:09:28.657036+0000","last_peered":"2024-09-16T11:09:28.657036+0000","last_clean":"2024-09-16T11:09:28.657036+0000","last_became_active":"2024-09-16T11:08:07.855380+0000","last_became_peered":"2024-09-16T11:08:07.855380+0000","last_unstale":"2024-09-16T11:09:28.657036+0000","last_undegraded":"2024-09-16T11:09:28.657036+0000","last_fullsized":"2024-09-16T11:09:28.657036+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'104","last_scrub_stamp":"2024-09-16T11:09:28.656884+0000","last_deep_scrub":"34'104","last_deep_scrub_stamp":"2024-09-16T11:09:28.656884+0000","last_clean_scrub_stamp":"2024-09-16T11:09:28.656884+0000","objects_scrubbed":0,"log_size":104,"log_dups_size":0,"ondisk_log_size":104,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T20:36:14.616243+0000","scrub_duration":0.0014366660000000001,"objects_trimmed":0,"snaptrim_duration":0.001457519,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1288,"num_read_kb":4433,"num_write":108,"num_write_kb":1673,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"35'73","reported_seq":426,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-16T11:09:11.792421+0000","last_change":"2024-09-16T11:09:11.792370+0000","last_active":"2024-09-16T11:09:11.792421+0000","last_peered":"2024-09-16T11:09:11.792421+0000","last_clean":"2024-09-16T11:09:11.792421+0000","last_became_active":"2024-09-16T11:08:07.859197+0000","last_became_peered":"2024-09-16T11:08:07.859197+0000","last_unstale":"2024-09-16T11:09:11.792421+0000","last_undegraded":"2024-09-16T11:09:11.792421+0000","last_fullsized":"2024-09-16T11:09:11.792421+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"35'73","last_scrub_stamp":"2024-09-16T11:09:11.792260+0000","last_deep_scrub":"35'73","last_deep_scrub_stamp":"2024-09-16T11:09:11.792260+0000","last_clean_scrub_stamp":"2024-09-16T11:09:11.792260+0000","objects_scrubbed":0,"log_size":73,"log_dups_size":0,"ondisk_log_size":73,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:29:15.052789+0000","scrub_duration":0.001460004,"objects_trimmed":0,"snaptrim_duration":0.00087272100000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":617,"num_read_kb":1509,"num_write":86,"num_write_kb":959,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.d","version":"34'78","reported_seq":483,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-16T11:09:29.637726+0000","last_change":"2024-09-16T11:09:29.637694+0000","last_active":"2024-09-16T11:09:29.637726+0000","last_peered":"2024-09-16T11:09:29.637726+0000","last_clean":"2024-09-16T11:09:29.637726+0000","last_became_active":"2024-09-16T11:08:07.853865+0000","last_became_peered":"2024-09-16T11:08:07.853865+0000","last_unstale":"2024-09-16T11:09:29.637726+0000","last_undegraded":"2024-09-16T11:09:29.637726+0000","last_fullsized":"2024-09-16T11:09:29.637726+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'78","last_scrub_stamp":"2024-09-16T11:09:29.637604+0000","last_deep_scrub":"34'78","last_deep_scrub_stamp":"2024-09-16T11:09:29.637604+0000","last_clean_scrub_stamp":"2024-09-16T11:09:29.637604+0000","objects_scrubbed":0,"log_size":78,"log_dups_size":0,"ondisk_log_size":78,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T19:18:23.814335+0000","scrub_duration":0.0013750100000000001,"objects_trimmed":0,"snaptrim_duration":0.001054501,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":785,"num_read_kb":1332,"num_write":70,"num_write_kb":661,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.e","version":"34'55","reported_seq":340,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-16T11:09:10.614689+0000","last_change":"2024-09-16T11:09:10.614661+0000","last_active":"2024-09-16T11:09:10.614689+0000","last_peered":"2024-09-16T11:09:10.614689+0000","last_clean":"2024-09-16T11:09:10.614689+0000","last_became_active":"2024-09-16T11:08:07.847486+0000","last_became_peered":"2024-09-16T11:08:07.847486+0000","last_unstale":"2024-09-16T11:09:10.614689+0000","last_undegraded":"2024-09-16T11:09:10.614689+0000","last_fullsized":"2024-09-16T11:09:10.614689+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'55","last_scrub_stamp":"2024-09-16T11:09:10.614573+0000","last_deep_scrub":"34'55","last_deep_scrub_stamp":"2024-09-16T11:09:10.614573+0000","last_clean_scrub_stamp":"2024-09-16T11:09:10.614573+0000","objects_scrubbed":0,"log_size":55,"log_dups_size":0,"ondisk_log_size":55,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:29:56.801023+0000","scrub_duration":0.001406416,"objects_trimmed":0,"snaptrim_duration":0.001186241,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":234,"num_read_kb":2061,"num_write":49,"num_write_kb":758,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"29'29","reported_seq":245,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-16T11:09:18.785549+0000","last_change":"2024-09-16T11:09:18.785526+0000","last_active":"2024-09-16T11:09:18.785549+0000","last_peered":"2024-09-16T11:09:18.785549+0000","last_clean":"2024-09-16T11:09:18.785549+0000","last_became_active":"2024-09-16T11:08:07.859480+0000","last_became_peered":"2024-09-16T11:08:07.859480+0000","last_unstale":"2024-09-16T11:09:18.785549+0000","last_undegraded":"2024-09-16T11:09:18.785549+0000","last_fullsized":"2024-09-16T11:09:18.785549+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"29'29","last_scrub_stamp":"2024-09-16T11:09:18.785446+0000","last_deep_scrub":"29'29","last_deep_scrub_stamp":"2024-09-16T11:09:18.785446+0000","last_clean_scrub_stamp":"2024-09-16T11:09:18.785446+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T13:49:26.458203+0000","scrub_duration":0.001225851,"objects_trimmed":0,"snaptrim_duration":0.001035411,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":155,"num_read_kb":866,"num_write":30,"num_write_kb":312,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"33'33","reported_seq":185,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-16T11:09:34.716712+0000","last_change":"2024-09-16T11:09:34.716662+0000","last_active":"2024-09-16T11:09:34.716712+0000","last_peered":"2024-09-16T11:09:34.716712+0000","last_clean":"2024-09-16T11:09:34.716712+0000","last_became_active":"2024-09-16T11:08:07.855591+0000","last_became_peered":"2024-09-16T11:08:07.855591+0000","last_unstale":"2024-09-16T11:09:34.716712+0000","last_undegraded":"2024-09-16T11:09:34.716712+0000","last_fullsized":"2024-09-16T11:09:34.716712+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'33","last_scrub_stamp":"2024-09-16T11:09:34.716556+0000","last_deep_scrub":"33'33","last_deep_scrub_stamp":"2024-09-16T11:09:34.716556+0000","last_clean_scrub_stamp":"2024-09-16T11:09:34.716556+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":0,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T18:01:16.643839+0000","scrub_duration":0.0014856559999999999,"objects_trimmed":0,"snaptrim_duration":0.00045890699999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":83,"num_read_kb":512,"num_write":38,"num_write_kb":467,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.11","version":"33'88","reported_seq":972,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-16T11:09:09.612356+0000","last_change":"2024-09-16T11:09:09.612308+0000","last_active":"2024-09-16T11:09:09.612356+0000","last_peered":"2024-09-16T11:09:09.612356+0000","last_clean":"2024-09-16T11:09:09.612356+0000","last_became_active":"2024-09-16T11:08:07.856626+0000","last_became_peered":"2024-09-16T11:08:07.856626+0000","last_unstale":"2024-09-16T11:09:09.612356+0000","last_undegraded":"2024-09-16T11:09:09.612356+0000","last_fullsized":"2024-09-16T11:09:09.612356+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'88","last_scrub_stamp":"2024-09-16T11:09:09.612205+0000","last_deep_scrub":"33'88","last_deep_scrub_stamp":"2024-09-16T11:09:09.612205+0000","last_clean_scrub_stamp":"2024-09-16T11:09:09.612205+0000","objects_scrubbed":0,"log_size":88,"log_dups_size":0,"ondisk_log_size":88,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T14:12:04.711301+0000","scrub_duration":0.0014779610000000001,"objects_trimmed":0,"snaptrim_duration":0.001031704,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1527,"num_read_kb":3994,"num_write":85,"num_write_kb":292,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"34'190","reported_seq":392,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-16T11:09:43.767176+0000","last_change":"2024-09-16T11:09:43.767134+0000","last_active":"2024-09-16T11:09:43.767176+0000","last_peered":"2024-09-16T11:09:43.767176+0000","last_clean":"2024-09-16T11:09:43.767176+0000","last_became_active":"2024-09-16T11:08:07.857698+0000","last_became_peered":"2024-09-16T11:08:07.857698+0000","last_unstale":"2024-09-16T11:09:43.767176+0000","last_undegraded":"2024-09-16T11:09:43.767176+0000","last_fullsized":"2024-09-16T11:09:43.767176+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'190","last_scrub_stamp":"2024-09-16T11:09:43.767025+0000","last_deep_scrub":"34'190","last_deep_scrub_stamp":"2024-09-16T11:09:43.767025+0000","last_clean_scrub_stamp":"2024-09-16T11:09:43.767025+0000","objects_scrubbed":0,"log_size":190,"log_dups_size":0,"ondisk_log_size":190,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:43:09.835253+0000","scrub_duration":0.001645182,"objects_trimmed":0,"snaptrim_duration":0.00040617399999999998,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":912,"num_read_kb":991,"num_write":288,"num_write_kb":407,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.13","version":"34'232","reported_seq":707,"reported_epoch":37,"state":"active+clean","last_fresh":"2024-09-16T11:09:38.740006+0000","last_change":"2024-09-16T11:09:38.739980+0000","last_active":"2024-09-16T11:09:38.740006+0000","last_peered":"2024-09-16T11:09:38.740006+0000","last_clean":"2024-09-16T11:09:38.740006+0000","last_became_active":"2024-09-16T11:08:07.853749+0000","last_became_peered":"2024-09-16T11:08:07.853749+0000","last_unstale":"2024-09-16T11:09:38.740006+0000","last_undegraded":"2024-09-16T11:09:38.740006+0000","last_fullsized":"2024-09-16T11:09:38.740006+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"34'232","last_scrub_stamp":"2024-09-16T11:09:38.739895+0000","last_deep_scrub":"34'232","last_deep_scrub_stamp":"2024-09-16T11:09:38.739895+0000","last_clean_scrub_stamp":"2024-09-16T11:09:38.739895+0000","objects_scrubbed":0,"log_size":232,"log_dups_size":0,"ondisk_log_size":232,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T17:43:04.808124+0000","scrub_duration":0.0013537639999999999,"objects_trimmed":0,"snaptrim_duration":0.00063776800000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1659,"num_read_kb":2488,"num_write":313,"num_write_kb":1774,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":16005,"num_read_kb":49038,"num_write":3041,"num_write_kb":27918,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2485,"ondisk_log_size":2485,"up":64,"acting":64,"num_store_stats":3},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5,"ondisk_log_size":5,"up":16,"acting":16,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":56,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":54,"ondisk_log_size":54,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":2,"up_from":8,"seq":34359738399,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":305976,"kb_used_data":304,"kb_used_omap":1,"kb_used_meta":305662,"kb_avail":94065864,"statfs":{"total":96636764160,"available":96323444736,"internally_reserved":0,"allocated":311296,"data_stored":120927,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":312998232},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 11:09:40 2024","interfaces":[{"interface":"back","average":{"1min":0.26400000000000001,"5min":0.22900000000000001,"15min":0.223},"min":{"1min":0.13700000000000001,"5min":0.13700000000000001,"15min":0.13700000000000001},"max":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"last":0.30599999999999999},{"interface":"front","average":{"1min":0.28100000000000003,"5min":0.24099999999999999,"15min":0.23499999999999999},"min":{"1min":0.19700000000000001,"5min":0.17000000000000001,"15min":0.17000000000000001},"max":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"last":0.33400000000000002}]},{"osd":1,"last update":"Mon Sep 16 11:09:40 2024","interfaces":[{"interface":"back","average":{"1min":0.30299999999999999,"5min":0.251,"15min":0.24199999999999999},"min":{"1min":0.16700000000000001,"5min":0.159,"15min":0.159},"max":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.32300000000000001},{"interface":"front","average":{"1min":0.30199999999999999,"5min":0.26200000000000001,"15min":0.255},"min":{"1min":0.17799999999999999,"5min":0.17799999999999999,"15min":0.17799999999999999},"max":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"last":0.35499999999999998}]}]},{"osd":1,"up_from":8,"seq":34359738400,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":306564,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":305662,"kb_avail":94065276,"statfs":{"total":96636764160,"available":96322842624,"internally_reserved":0,"allocated":913408,"data_stored":712075,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":312998232},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Sep 16 11:09:38 2024","interfaces":[{"interface":"back","average":{"1min":0.29199999999999998,"5min":0.26500000000000001,"15min":0.26100000000000001},"min":{"1min":0.17599999999999999,"5min":0.17399999999999999,"15min":0.17399999999999999},"max":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"last":0.26600000000000001},{"interface":"front","average":{"1min":0.27200000000000002,"5min":0.26600000000000001,"15min":0.26500000000000001},"min":{"1min":0.183,"5min":0.17599999999999999,"15min":0.17599999999999999},"max":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"last":0.32700000000000001}]},{"osd":2,"last update":"Mon Sep 16 11:09:38 2024","interfaces":[{"interface":"back","average":{"1min":0.28799999999999998,"5min":0.27400000000000002,"15min":0.27200000000000002},"min":{"1min":0.183,"5min":0.16600000000000001,"15min":0.16600000000000001},"max":{"1min":0.46500000000000002,"5min":0.48899999999999999,"15min":0.48899999999999999},"last":0.28699999999999998},{"interface":"front","average":{"1min":0.29699999999999999,"5min":0.28199999999999997,"15min":0.28000000000000003},"min":{"1min":0.20000000000000001,"5min":0.191,"15min":0.191},"max":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"last":0.311}]}]},{"osd":0,"up_from":8,"seq":34359738399,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":306556,"kb_used_data":884,"kb_used_omap":1,"kb_used_meta":305662,"kb_avail":94065284,"statfs":{"total":96636764160,"available":96322850816,"internally_reserved":0,"allocated":905216,"data_stored":711295,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":312998232},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Sep 16 11:09:42 2024","interfaces":[{"interface":"back","average":{"1min":0.27500000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"min":{"1min":0.159,"5min":0.14599999999999999,"15min":0.14599999999999999},"max":{"1min":0.42099999999999999,"5min":0.51800000000000002,"15min":0.51800000000000002},"last":0.19400000000000001},{"interface":"front","average":{"1min":0.28499999999999998,"5min":0.30499999999999999,"15min":0.308},"min":{"1min":0.17799999999999999,"5min":0.16600000000000001,"15min":0.16600000000000001},"max":{"1min":0.40999999999999998,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.21199999999999999}]},{"osd":2,"last update":"Mon Sep 16 11:09:42 2024","interfaces":[{"interface":"back","average":{"1min":0.29599999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"min":{"1min":0.19500000000000001,"5min":0.19500000000000001,"15min":0.19500000000000001},"max":{"1min":0.42599999999999999,"5min":0.46700000000000003,"15min":0.46700000000000003},"last":0.317},{"interface":"front","average":{"1min":0.315,"5min":0.31,"15min":0.309},"min":{"1min":0.19900000000000001,"5min":0.186,"15min":0.186},"max":{"1min":0.45500000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"last":0.30199999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T11:09:59.266 DEBUG:teuthology.orchestra.run.smithi102:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-09-16T11:09:59.539 INFO:teuthology.misc:Shutting down mds daemons... 2024-09-16T11:09:59.539 INFO:teuthology.misc:Shutting down osd daemons... 2024-09-16T11:09:59.539 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-09-16T11:09:59.539 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T11:09:59.575 INFO:tasks.ceph.osd.0:Stopped 2024-09-16T11:09:59.575 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-09-16T11:09:59.575 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T11:09:59.651 INFO:tasks.ceph.osd.1:Stopped 2024-09-16T11:09:59.651 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-09-16T11:09:59.651 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T11:09:59.726 INFO:tasks.ceph.osd.2:Stopped 2024-09-16T11:09:59.726 INFO:teuthology.misc:Shutting down mgr daemons... 2024-09-16T11:09:59.727 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-09-16T11:09:59.727 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T11:09:59.790 INFO:tasks.ceph.mgr.x:Stopped 2024-09-16T11:09:59.790 INFO:teuthology.misc:Shutting down mon daemons... 2024-09-16T11:09:59.791 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-09-16T11:09:59.792 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T11:09:59.841 INFO:tasks.ceph.mon.a:Stopped 2024-09-16T11:09:59.842 INFO:tasks.ceph:Checking cluster log for badness... 2024-09-16T11:09:59.842 DEBUG:teuthology.orchestra.run.smithi102:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | head -n 1 2024-09-16T11:09:59.910 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi102.front.sepia.ceph.com 2024-09-16T11:09:59.910 DEBUG:teuthology.orchestra.run.smithi102:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-09-16T11:10:00.104 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi102.front.sepia.ceph.com 2024-09-16T11:10:00.104 DEBUG:teuthology.orchestra.run.smithi102:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-09-16T11:10:00.193 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi102.front.sepia.ceph.com 2024-09-16T11:10:00.193 DEBUG:teuthology.orchestra.run.smithi102:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-09-16T11:10:00.278 INFO:tasks.ceph:Archiving mon data... 2024-09-16T11:10:00.280 DEBUG:teuthology.misc:Transferring archived files from smithi102:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/sangadi-2024-09-16_10:44:09-rbd-wip-bharath9-testing-2024-09-13-1444-quincy-distro-default-smithi/7907146/data/mon.a.tgz 2024-09-16T11:10:00.280 DEBUG:teuthology.orchestra.run.smithi102:> mktemp 2024-09-16T11:10:00.295 INFO:teuthology.orchestra.run.smithi102.stdout:/tmp/tmp.Tw6vCyp2C5 2024-09-16T11:10:00.295 DEBUG:teuthology.orchestra.run.smithi102:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.Tw6vCyp2C5 2024-09-16T11:10:00.440 DEBUG:teuthology.orchestra.run.smithi102:> sudo chmod 0666 /tmp/tmp.Tw6vCyp2C5 2024-09-16T11:10:00.524 DEBUG:teuthology.orchestra.remote:smithi102:/tmp/tmp.Tw6vCyp2C5 is 174KB 2024-09-16T11:10:00.684 DEBUG:teuthology.orchestra.run.smithi102:> rm -fr /tmp/tmp.Tw6vCyp2C5 2024-09-16T11:10:00.717 INFO:tasks.ceph:Cleaning ceph cluster... 2024-09-16T11:10:00.717 DEBUG:teuthology.orchestra.run.smithi102:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-09-16T11:10:00.793 INFO:tasks.ceph:Checking for errors in any valgrind logs... 2024-09-16T11:10:00.793 DEBUG:teuthology.orchestra.run.smithi102:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2024-09-16T11:10:00.867 INFO:tasks.ceph:Archiving crash dumps... 2024-09-16T11:10:01.102 DEBUG:teuthology.misc:Transferring archived files from smithi102:/var/lib/ceph/crash to /home/teuthworker/archive/sangadi-2024-09-16_10:44:09-rbd-wip-bharath9-testing-2024-09-13-1444-quincy-distro-default-smithi/7907146/remote/smithi102/crash 2024-09-16T11:10:01.109 DEBUG:teuthology.orchestra.run.smithi102:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-09-16T11:10:01.134 INFO:tasks.ceph:Compressing logs... 2024-09-16T11:10:01.134 DEBUG:teuthology.orchestra.run.smithi102:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-16T11:10:01.224 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2024-09-16T11:10:01.224 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.27728.log 2024-09-16T11:10:01.224 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38470.log 2024-09-16T11:10:01.224 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph.tmp-client.admin.27728.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2024-09-16T11:10:01.224 INFO:teuthology.orchestra.run.smithi102.stderr: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.27728.log.gz 2024-09-16T11:10:01.224 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.38470.log: 0.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.38470.log.gz 2024-09-16T11:10:01.225 INFO:teuthology.orchestra.run.smithi102.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.38165.log 2024-09-16T11:10:01.225 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37635.log 2024-09-16T11:10:01.225 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.38165.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38165.log.gz 2024-09-16T11:10:01.225 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37551.log 2024-09-16T11:10:01.225 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38596.log 2024-09-16T11:10:01.225 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.37635.log: /var/log/ceph/ceph-client.admin.37551.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37635.log.gz 2024-09-16T11:10:01.225 INFO:teuthology.orchestra.run.smithi102.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37551.log.gz 2024-09-16T11:10:01.225 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38313.log 2024-09-16T11:10:01.225 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37931.log 2024-09-16T11:10:01.225 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.38596.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38596.log.gz 2024-09-16T11:10:01.225 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.38313.log: gzip -5 --verbose -- 0.0% /var/log/ceph/ceph-client.admin.38164.log 2024-09-16T11:10:01.226 INFO:teuthology.orchestra.run.smithi102.stderr: -- replaced with /var/log/ceph/ceph-client.admin.38313.log.gz 2024-09-16T11:10:01.226 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.37931.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37931.log.gz 2024-09-16T11:10:01.227 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38848.log 2024-09-16T11:10:01.227 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38185.log 2024-09-16T11:10:01.227 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.38164.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38164.log.gz 2024-09-16T11:10:01.227 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.38848.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-09-16T11:10:01.227 INFO:teuthology.orchestra.run.smithi102.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38848.log.gz 2024-09-16T11:10:01.227 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.38185.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38764.log 2024-09-16T11:10:01.227 INFO:teuthology.orchestra.run.smithi102.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38185.log.gz 2024-09-16T11:10:01.227 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37809.log 2024-09-16T11:10:01.227 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.38764.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38764.log.gz 2024-09-16T11:10:01.227 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38512.log 2024-09-16T11:10:01.227 INFO:teuthology.orchestra.run.smithi102.stderr: 86.8% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-09-16T11:10:01.227 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39100.log 2024-09-16T11:10:01.227 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.37809.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37809.log.gz 2024-09-16T11:10:01.228 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.38512.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32057.log 2024-09-16T11:10:01.228 INFO:teuthology.orchestra.run.smithi102.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38512.log.gz 2024-09-16T11:10:01.228 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.39100.log: gzip -5 0.0% --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.39100.log.gz 2024-09-16T11:10:01.228 INFO:teuthology.orchestra.run.smithi102.stderr: /var/log/ceph/ceph-client.admin.38019.log 2024-09-16T11:10:01.228 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32255.log 2024-09-16T11:10:01.228 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.32057.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32057.log.gz 2024-09-16T11:10:01.228 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.38019.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37767.log 2024-09-16T11:10:01.228 INFO:teuthology.orchestra.run.smithi102.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38019.log.gz 2024-09-16T11:10:01.228 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.32255.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32255.log.gz 2024-09-16T11:10:01.228 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38806.log 2024-09-16T11:10:01.228 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32394.log 2024-09-16T11:10:01.228 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.37767.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37767.log.gz 2024-09-16T11:10:01.228 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.38806.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38806.log.gz 2024-09-16T11:10:01.229 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2024-09-16T11:10:01.229 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.32394.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.38378.log 2024-09-16T11:10:01.229 INFO:teuthology.orchestra.run.smithi102.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32394.log.gz 2024-09-16T11:10:01.229 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38059.log 2024-09-16T11:10:01.229 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.0.38378.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38554.log 2024-09-16T11:10:01.229 INFO:teuthology.orchestra.run.smithi102.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.38378.log.gz 2024-09-16T11:10:01.229 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.38059.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38058.log 2024-09-16T11:10:01.229 INFO:teuthology.orchestra.run.smithi102.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38059.log.gz 2024-09-16T11:10:01.229 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.38554.log: gzip 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38554.log.gz 2024-09-16T11:10:01.229 INFO:teuthology.orchestra.run.smithi102.stderr: -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2024-09-16T11:10:01.229 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.38058.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38058.log.gz 2024-09-16T11:10:01.229 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/valgrind/fsx_client.0.log 2024-09-16T11:10:01.229 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38890.log 2024-09-16T11:10:01.230 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/valgrind/fsx_client.0.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-09-16T11:10:01.230 INFO:teuthology.orchestra.run.smithi102.stderr: 63.1% -- replaced with /var/log/ceph/valgrind/fsx_client.0.log.gz 2024-09-16T11:10:01.230 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.38890.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38890.log.gz 2024-09-16T11:10:01.230 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38638.log 2024-09-16T11:10:01.230 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39060.log 2024-09-16T11:10:01.230 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.38638.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38638.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2024-09-16T11:10:01.230 INFO:teuthology.orchestra.run.smithi102.stderr: 83.1% -- replaced with /var/log/ceph/ceph.log.gz 2024-09-16T11:10:01.230 INFO:teuthology.orchestra.run.smithi102.stderr: 2024-09-16T11:10:01.230 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.39060.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39060.log.gz 2024-09-16T11:10:01.230 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37932.log 2024-09-16T11:10:01.230 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32462.log 2024-09-16T11:10:01.230 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.37932.log: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38932.log 2024-09-16T11:10:01.230 INFO:teuthology.orchestra.run.smithi102.stderr: -- replaced with /var/log/ceph/ceph-client.admin.37932.log.gz 2024-09-16T11:10:01.230 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38680.log 2024-09-16T11:10:01.231 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.32462.log: /var/log/ceph/ceph-client.admin.38932.log: 0.0% 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32462.log.gz 2024-09-16T11:10:01.231 INFO:teuthology.orchestra.run.smithi102.stderr: -- replaced with /var/log/ceph/ceph-client.admin.38932.log.gz 2024-09-16T11:10:01.231 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38974.log 2024-09-16T11:10:01.231 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.38680.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38680.log.gz 2024-09-16T11:10:01.231 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37691.log 2024-09-16T11:10:01.231 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.38974.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38974.log.gz 2024-09-16T11:10:01.231 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39016.log 2024-09-16T11:10:01.231 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38355.log 2024-09-16T11:10:01.231 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.37691.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37691.log.gz 2024-09-16T11:10:01.232 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.39016.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39016.log.gz 2024-09-16T11:10:01.232 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37933.log 2024-09-16T11:10:01.232 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.38355.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38355.log.gz 2024-09-16T11:10:01.232 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37851.log 2024-09-16T11:10:01.232 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38722.log 2024-09-16T11:10:01.233 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.37933.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37933.log.gz 2024-09-16T11:10:01.233 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.37851.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37851.log.gz 2024-09-16T11:10:01.233 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32215.log 2024-09-16T11:10:01.233 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.38722.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38722.log.gz 2024-09-16T11:10:01.233 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38271.log 2024-09-16T11:10:01.233 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37716.log 2024-09-16T11:10:01.233 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.32215.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32215.log.gz 2024-09-16T11:10:01.233 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.38271.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38271.log.gz 2024-09-16T11:10:01.234 INFO:teuthology.orchestra.run.smithi102.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32295.log 2024-09-16T11:10:01.234 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.37716.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37716.log.gz 2024-09-16T11:10:01.234 INFO:teuthology.orchestra.run.smithi102.stderr:/var/log/ceph/ceph-client.admin.32295.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32295.log.gz 2024-09-16T11:10:01.237 INFO:teuthology.orchestra.run.smithi102.stderr: 91.4% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2024-09-16T11:10:01.292 INFO:teuthology.orchestra.run.smithi102.stderr: 90.6% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2024-09-16T11:10:01.656 INFO:teuthology.orchestra.run.smithi102.stderr: 94.6% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2024-09-16T11:10:01.703 INFO:teuthology.orchestra.run.smithi102.stderr: 94.8% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2024-09-16T11:10:01.776 INFO:teuthology.orchestra.run.smithi102.stderr: 94.6% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2024-09-16T11:10:01.778 INFO:teuthology.orchestra.run.smithi102.stderr: 2024-09-16T11:10:01.778 INFO:teuthology.orchestra.run.smithi102.stderr:real 0m0.589s 2024-09-16T11:10:01.778 INFO:teuthology.orchestra.run.smithi102.stderr:user 0m1.599s 2024-09-16T11:10:01.778 INFO:teuthology.orchestra.run.smithi102.stderr:sys 0m0.095s 2024-09-16T11:10:01.779 INFO:tasks.ceph:Archiving logs... 2024-09-16T11:10:01.779 DEBUG:teuthology.misc:Transferring archived files from smithi102:/var/log/ceph to /home/teuthworker/archive/sangadi-2024-09-16_10:44:09-rbd-wip-bharath9-testing-2024-09-13-1444-quincy-distro-default-smithi/7907146/remote/smithi102/log 2024-09-16T11:10:01.780 DEBUG:teuthology.orchestra.run.smithi102:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-16T11:10:02.118 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-09-16T11:10:02.126 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-09-16T11:10:02.127 DEBUG:teuthology.orchestra.run.smithi102:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-16T11:10:02.180 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-09-16T11:10:02.188 INFO:teuthology.task.clock:Checking final clock skew... 2024-09-16T11:10:02.189 DEBUG:teuthology.orchestra.run.smithi102:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-16T11:10:02.209 INFO:teuthology.orchestra.run.smithi102.stderr:bash: line 1: ntpq: command not found 2024-09-16T11:10:02.214 INFO:teuthology.orchestra.run.smithi102.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-16T11:10:02.214 INFO:teuthology.orchestra.run.smithi102.stdout:=============================================================================== 2024-09-16T11:10:02.214 INFO:teuthology.orchestra.run.smithi102.stdout:^* hv01.front.sepia.ceph.com 3 6 377 20 -560us[ -559us] +/- 52ms 2024-09-16T11:10:02.214 INFO:teuthology.orchestra.run.smithi102.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 19 +1381us[+1381us] +/- 57ms 2024-09-16T11:10:02.214 INFO:teuthology.orchestra.run.smithi102.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 21 -793us[ -791us] +/- 60ms 2024-09-16T11:10:02.214 INFO:teuthology.orchestra.run.smithi102.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-09-16T11:10:02.214 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-09-16T11:10:02.227 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-09-16T11:10:02.227 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-09-16T11:10:02.314 DEBUG:teuthology.orchestra.run.smithi102:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-16T11:10:02.376 DEBUG:teuthology.orchestra.run.smithi102:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2024-09-16T11:10:02.403 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T11:10:02.403 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-09-16T11:10:02.413 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-09-16T11:10:02.470 INFO:teuthology.task.internal:Duration was 892.602965 seconds 2024-09-16T11:10:02.470 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-09-16T11:10:02.545 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-09-16T11:10:02.546 DEBUG:teuthology.orchestra.run.smithi102:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-16T11:10:02.584 INFO:teuthology.orchestra.run.smithi102.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-16T11:10:02.625 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-09-16T11:10:02.625 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi102.front.sepia.ceph.com 2024-09-16T11:10:02.625 DEBUG:teuthology.orchestra.run.smithi102:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-16T11:10:02.686 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-09-16T11:10:02.686 DEBUG:teuthology.orchestra.run.smithi102:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-16T11:10:02.751 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-09-16T11:10:02.752 DEBUG:teuthology.orchestra.run.smithi102:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-16T11:10:02.893 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-09-16T11:10:02.903 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-09-16T11:10:02.903 DEBUG:teuthology.orchestra.run.smithi102:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-16T11:10:02.928 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-09-16T11:10:02.976 DEBUG:teuthology.orchestra.run.smithi102:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-16T11:10:02.999 INFO:teuthology.orchestra.run.smithi102.stdout:kernel.core_pattern = core 2024-09-16T11:10:03.035 DEBUG:teuthology.orchestra.run.smithi102:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-16T11:10:03.090 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T11:10:03.090 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-09-16T11:10:03.100 INFO:teuthology.task.internal:Transferring archived files... 2024-09-16T11:10:03.100 DEBUG:teuthology.misc:Transferring archived files from smithi102:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/sangadi-2024-09-16_10:44:09-rbd-wip-bharath9-testing-2024-09-13-1444-quincy-distro-default-smithi/7907146/remote/smithi102 2024-09-16T11:10:03.101 DEBUG:teuthology.orchestra.run.smithi102:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-16T11:10:03.165 INFO:teuthology.task.internal:Removing archive directory... 2024-09-16T11:10:03.166 DEBUG:teuthology.orchestra.run.smithi102:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-16T11:10:03.214 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-09-16T11:10:03.225 INFO:teuthology.task.internal:Not uploading archives. 2024-09-16T11:10:03.225 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-09-16T11:10:03.272 INFO:teuthology.task.internal:Tidying up after the test... 2024-09-16T11:10:03.273 DEBUG:teuthology.orchestra.run.smithi102:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-16T11:10:03.288 INFO:teuthology.orchestra.run.smithi102.stdout: 265324 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 16 11:10 /home/ubuntu/cephtest 2024-09-16T11:10:03.289 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-09-16T11:10:03.337 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-09-16T11:10:03.464 INFO:teuthology.run:Summary data: description: rbd/valgrind/{base/install centos_latest clusters/{fixed-1 openstack} conf/{disable-pool-app} objectstore/bluestore-hybrid validator/memcheck workloads/fsx} duration: 892.6029646396637 flavor: default owner: scheduled_sangadi@teuthology success: true 2024-09-16T11:10:03.465 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-16T11:10:03.545 INFO:teuthology.run:pass