2024-09-18T10:21:12.672 INFO:root:teuthology version: 1.2.2.dev2+g3752d38 2024-09-18T10:21:12.688 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-18T10:21:12.771 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/skanta-2024-09-18_09:13:00-rados-wip-bharath11-testing-2024-09-18-1115-quincy-distro-default-smithi/7910878 branch: wip-bharath11-testing-2024-09-18-1115-quincy description: rados/singleton-nomsgr/{all/health-warnings mon_election/classic rados supported-random-distro$/{ubuntu_latest}} email: skanta@redhat.com first_in_suite: false job_id: '7910878' kernel: kdb: 1 sha1: distro ktype: distro last_in_suite: false machine_type: smithi name: skanta-2024-09-18_09:13:00-rados-wip-bharath11-testing-2024-09-18-1115-quincy-distro-default-smithi no_nested_subset: false os_type: ubuntu os_version: '20.04' overrides: admin_socket: branch: wip-bharath11-testing-2024-09-18-1115-quincy ceph: conf: global: mon election default strategy: 1 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon scrub interval: 300 osd: bluestore zero block detection: true debug ms: 1 debug osd: 20 osd debug verify cached snaps: true osd debug verify missing on start: true osd mclock override recovery settings: true osd mclock profile: high_recovery_ops osd op queue: debug_random osd op queue cut off: debug_random flavor: default log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) sha1: 7061ea6ca50908d104e9f3979dc615e118d9e153 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} install: ceph: flavor: default sha1: 7061ea6ca50908d104e9f3979dc615e118d9e153 workunit: branch: wip-bharath11-testing-2024-09-18-1115-quincy sha1: 7061ea6ca50908d104e9f3979dc615e118d9e153 owner: scheduled_skanta@teuthology priority: 99 repo: https://github.com/ceph/ceph-ci.git roles: - - mon.a - mgr.x - osd.0 - osd.1 - osd.2 - osd.3 - osd.4 - osd.5 - osd.6 - osd.7 - osd.8 - osd.9 - client.0 seed: 4631 sha1: 7061ea6ca50908d104e9f3979dc615e118d9e153 sleep_before_teardown: 0 subset: 111/120000 suite: rados suite_branch: wip-bharath11-testing-2024-09-18-1115-quincy suite_path: /home/teuthworker/src/github.com_ceph_ceph-c_7061ea6ca50908d104e9f3979dc615e118d9e153/qa suite_relpath: qa suite_repo: https://github.com/ceph/ceph-ci.git suite_sha1: 7061ea6ca50908d104e9f3979dc615e118d9e153 targets: smithi134.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFyT/r7xuKfmg50w0OdQebr4A/ptJo1Zk6DyZNZNZM/q5rW+sznh+wvHqeT3e4YtCdZLaXyvMms8o8u8H5nXOYM= tasks: - install: null - ceph: conf: osd: osd max object name len: 400 osd max object namespace len: 64 log-ignorelist: - but it is still running - overall HEALTH_ - \(OSDMAP_FLAGS\) - \(OSD_ - \(PG_ - \(POOL_APP_NOT_ENABLED\) pre-mgr-commands: - sudo ceph config set mgr mgr_pool false --force - workunit: clients: all: - rados/test_health_warnings.sh teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: 3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a timestamp: 2024-09-18_09:13:00 tube: smithi user: skanta verbose: true worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856 2024-09-18T10:21:12.771 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_ceph_ceph-c_7061ea6ca50908d104e9f3979dc615e118d9e153/qa; will attempt to use it 2024-09-18T10:21:12.772 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_ceph_ceph-c_7061ea6ca50908d104e9f3979dc615e118d9e153/qa/tasks 2024-09-18T10:21:12.772 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-09-18T10:21:12.773 INFO:teuthology.task.internal:Checking packages... 2024-09-18T10:21:12.795 INFO:teuthology.task.internal:Checking packages for os_type 'ubuntu', flavor 'default' and ceph hash '7061ea6ca50908d104e9f3979dc615e118d9e153' 2024-09-18T10:21:12.795 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-18T10:21:12.795 INFO:teuthology.packaging:ref: None 2024-09-18T10:21:12.795 INFO:teuthology.packaging:tag: None 2024-09-18T10:21:12.795 INFO:teuthology.packaging:branch: wip-bharath11-testing-2024-09-18-1115-quincy 2024-09-18T10:21:12.795 INFO:teuthology.packaging:sha1: 7061ea6ca50908d104e9f3979dc615e118d9e153 2024-09-18T10:21:12.795 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&ref=wip-bharath11-testing-2024-09-18-1115-quincy 2024-09-18T10:21:12.945 INFO:teuthology.task.internal:Found packages for ceph version 17.2.7-1645-g7061ea6c-1focal 2024-09-18T10:21:12.947 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-09-18T10:21:12.954 INFO:teuthology.task.internal:no buildpackages task found 2024-09-18T10:21:12.954 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-09-18T10:21:12.975 INFO:teuthology.task.internal:Saving configuration 2024-09-18T10:21:12.985 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-09-18T10:21:12.990 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-09-18T10:21:13.012 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi134.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/skanta-2024-09-18_09:13:00-rados-wip-bharath11-testing-2024-09-18-1115-quincy-distro-default-smithi/7910878', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '20.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-18 10:15:34.839217', 'locked_by': 'scheduled_skanta@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIGETOy3xONfbx7kUFCNiUtRLNfkcrcEsWzPdq6Np/6Bw'} 2024-09-18T10:21:13.012 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-09-18T10:21:13.018 INFO:teuthology.task.internal:roles: ubuntu@smithi134.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'osd.8', 'osd.9', 'client.0'] 2024-09-18T10:21:13.018 INFO:teuthology.run_tasks:Running task console_log... 2024-09-18T10:21:13.050 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7faac7ba7ac0>, signals=[15]) 2024-09-18T10:21:13.050 INFO:teuthology.run_tasks:Running task internal.connect... 2024-09-18T10:21:13.056 INFO:teuthology.task.internal:Opening connections... 2024-09-18T10:21:13.056 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi134.front.sepia.ceph.com 2024-09-18T10:21:13.057 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi134.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-18T10:21:13.156 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-09-18T10:21:13.163 DEBUG:teuthology.orchestra.run.smithi134:> uname -m 2024-09-18T10:21:13.279 INFO:teuthology.orchestra.run.smithi134.stdout:x86_64 2024-09-18T10:21:13.279 DEBUG:teuthology.orchestra.run.smithi134:> cat /etc/os-release 2024-09-18T10:21:13.286 INFO:teuthology.orchestra.run.smithi134.stdout:NAME="Ubuntu" 2024-09-18T10:21:13.286 INFO:teuthology.orchestra.run.smithi134.stdout:VERSION="20.04.4 LTS (Focal Fossa)" 2024-09-18T10:21:13.286 INFO:teuthology.orchestra.run.smithi134.stdout:ID=ubuntu 2024-09-18T10:21:13.286 INFO:teuthology.orchestra.run.smithi134.stdout:ID_LIKE=debian 2024-09-18T10:21:13.286 INFO:teuthology.orchestra.run.smithi134.stdout:PRETTY_NAME="Ubuntu 20.04.4 LTS" 2024-09-18T10:21:13.286 INFO:teuthology.orchestra.run.smithi134.stdout:VERSION_ID="20.04" 2024-09-18T10:21:13.286 INFO:teuthology.orchestra.run.smithi134.stdout:HOME_URL="https://www.ubuntu.com/" 2024-09-18T10:21:13.286 INFO:teuthology.orchestra.run.smithi134.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2024-09-18T10:21:13.286 INFO:teuthology.orchestra.run.smithi134.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2024-09-18T10:21:13.286 INFO:teuthology.orchestra.run.smithi134.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2024-09-18T10:21:13.286 INFO:teuthology.orchestra.run.smithi134.stdout:VERSION_CODENAME=focal 2024-09-18T10:21:13.287 INFO:teuthology.orchestra.run.smithi134.stdout:UBUNTU_CODENAME=focal 2024-09-18T10:21:13.287 INFO:teuthology.lock.ops:Updating smithi134.front.sepia.ceph.com on lock server 2024-09-18T10:21:13.309 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-09-18T10:21:13.316 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-09-18T10:21:13.324 INFO:teuthology.task.internal:Checking for old test directory... 2024-09-18T10:21:13.324 DEBUG:teuthology.orchestra.run.smithi134:> test '!' -e /home/ubuntu/cephtest 2024-09-18T10:21:13.329 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-09-18T10:21:13.336 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-09-18T10:21:13.336 DEBUG:teuthology.orchestra.run.smithi134:> test -z $(ls -A /var/lib/ceph) 2024-09-18T10:21:13.390 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-09-18T10:21:13.422 INFO:teuthology.run_tasks:Running task kernel... 2024-09-18T10:21:13.438 INFO:teuthology.task.kernel:normalize config orig: {'kdb': 1, 'sha1': 'distro'} 2024-09-18T10:21:13.438 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': 1, 'sha1': 'distro'}}, timeout 300 2024-09-18T10:21:13.438 DEBUG:teuthology.orchestra.run.smithi134:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-18T10:21:13.443 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-18T10:21:13.444 DEBUG:teuthology.orchestra.run.smithi134:> uname -r 2024-09-18T10:21:13.450 INFO:teuthology.orchestra.run.smithi134.stdout:5.4.0-124-generic 2024-09-18T10:21:13.450 INFO:teuthology.task.kernel:Running kernel on smithi134: 5.4.0-124-generic 2024-09-18T10:21:13.451 DEBUG:teuthology.orchestra.run.smithi134:> sudo apt-get clean 2024-09-18T10:21:13.558 DEBUG:teuthology.orchestra.run.smithi134:> sudo apt-get update 2024-09-18T10:21:13.752 INFO:teuthology.orchestra.run.smithi134.stdout:Hit:1 http://archive.ubuntu.com/ubuntu focal InRelease 2024-09-18T10:21:13.762 INFO:teuthology.orchestra.run.smithi134.stdout:Get:2 http://archive.ubuntu.com/ubuntu focal-updates InRelease [128 kB] 2024-09-18T10:21:13.826 INFO:teuthology.orchestra.run.smithi134.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal-backports InRelease [128 kB] 2024-09-18T10:21:13.862 INFO:teuthology.orchestra.run.smithi134.stdout:Get:4 http://security.ubuntu.com/ubuntu focal-security InRelease [128 kB] 2024-09-18T10:21:14.150 INFO:teuthology.orchestra.run.smithi134.stdout:Get:5 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 Packages [3,598 kB] 2024-09-18T10:21:14.306 INFO:teuthology.orchestra.run.smithi134.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal-updates/main i386 Packages [1,036 kB] 2024-09-18T10:21:14.320 INFO:teuthology.orchestra.run.smithi134.stdout:Get:7 http://archive.ubuntu.com/ubuntu focal-updates/main Translation-en [553 kB] 2024-09-18T10:21:14.324 INFO:teuthology.orchestra.run.smithi134.stdout:Get:8 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 c-n-f Metadata [17.7 kB] 2024-09-18T10:21:14.324 INFO:teuthology.orchestra.run.smithi134.stdout:Get:9 http://archive.ubuntu.com/ubuntu focal-updates/restricted amd64 Packages [3,279 kB] 2024-09-18T10:21:14.351 INFO:teuthology.orchestra.run.smithi134.stdout:Get:10 http://archive.ubuntu.com/ubuntu focal-updates/restricted i386 Packages [39.6 kB] 2024-09-18T10:21:14.351 INFO:teuthology.orchestra.run.smithi134.stdout:Get:11 http://archive.ubuntu.com/ubuntu focal-updates/restricted Translation-en [459 kB] 2024-09-18T10:21:14.354 INFO:teuthology.orchestra.run.smithi134.stdout:Get:12 http://archive.ubuntu.com/ubuntu focal-updates/restricted amd64 c-n-f Metadata [548 B] 2024-09-18T10:21:14.354 INFO:teuthology.orchestra.run.smithi134.stdout:Get:13 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 Packages [1,231 kB] 2024-09-18T10:21:14.362 INFO:teuthology.orchestra.run.smithi134.stdout:Get:14 http://archive.ubuntu.com/ubuntu focal-updates/universe i386 Packages [808 kB] 2024-09-18T10:21:14.368 INFO:teuthology.orchestra.run.smithi134.stdout:Get:15 http://archive.ubuntu.com/ubuntu focal-updates/universe Translation-en [295 kB] 2024-09-18T10:21:14.370 INFO:teuthology.orchestra.run.smithi134.stdout:Get:16 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 c-n-f Metadata [28.3 kB] 2024-09-18T10:21:14.370 INFO:teuthology.orchestra.run.smithi134.stdout:Get:17 http://archive.ubuntu.com/ubuntu focal-updates/multiverse amd64 Packages [27.1 kB] 2024-09-18T10:21:14.371 INFO:teuthology.orchestra.run.smithi134.stdout:Get:18 http://archive.ubuntu.com/ubuntu focal-updates/multiverse i386 Packages [8,440 B] 2024-09-18T10:21:14.372 INFO:teuthology.orchestra.run.smithi134.stdout:Get:19 http://archive.ubuntu.com/ubuntu focal-updates/multiverse Translation-en [7,936 B] 2024-09-18T10:21:14.372 INFO:teuthology.orchestra.run.smithi134.stdout:Get:20 http://archive.ubuntu.com/ubuntu focal-updates/multiverse amd64 c-n-f Metadata [616 B] 2024-09-18T10:21:14.389 INFO:teuthology.orchestra.run.smithi134.stdout:Get:21 http://archive.ubuntu.com/ubuntu focal-backports/main i386 Packages [36.1 kB] 2024-09-18T10:21:14.389 INFO:teuthology.orchestra.run.smithi134.stdout:Get:22 http://archive.ubuntu.com/ubuntu focal-backports/main amd64 Packages [45.7 kB] 2024-09-18T10:21:14.390 INFO:teuthology.orchestra.run.smithi134.stdout:Get:23 http://archive.ubuntu.com/ubuntu focal-backports/main amd64 c-n-f Metadata [1,420 B] 2024-09-18T10:21:14.390 INFO:teuthology.orchestra.run.smithi134.stdout:Get:24 http://archive.ubuntu.com/ubuntu focal-backports/universe amd64 Packages [25.0 kB] 2024-09-18T10:21:14.390 INFO:teuthology.orchestra.run.smithi134.stdout:Get:25 http://archive.ubuntu.com/ubuntu focal-backports/universe i386 Packages [13.8 kB] 2024-09-18T10:21:14.391 INFO:teuthology.orchestra.run.smithi134.stdout:Get:26 http://archive.ubuntu.com/ubuntu focal-backports/universe amd64 c-n-f Metadata [880 B] 2024-09-18T10:21:14.490 INFO:teuthology.orchestra.run.smithi134.stdout:Get:27 http://security.ubuntu.com/ubuntu focal-security/main i386 Packages [817 kB] 2024-09-18T10:21:14.827 INFO:teuthology.orchestra.run.smithi134.stdout:Get:28 http://security.ubuntu.com/ubuntu focal-security/main amd64 Packages [3,223 kB] 2024-09-18T10:21:15.021 INFO:teuthology.orchestra.run.smithi134.stdout:Get:29 http://security.ubuntu.com/ubuntu focal-security/main Translation-en [475 kB] 2024-09-18T10:21:15.035 INFO:teuthology.orchestra.run.smithi134.stdout:Get:30 http://security.ubuntu.com/ubuntu focal-security/main amd64 c-n-f Metadata [14.3 kB] 2024-09-18T10:21:15.035 INFO:teuthology.orchestra.run.smithi134.stdout:Get:31 http://security.ubuntu.com/ubuntu focal-security/restricted amd64 Packages [3,152 kB] 2024-09-18T10:21:15.126 INFO:teuthology.orchestra.run.smithi134.stdout:Get:32 http://security.ubuntu.com/ubuntu focal-security/restricted i386 Packages [38.3 kB] 2024-09-18T10:21:15.130 INFO:teuthology.orchestra.run.smithi134.stdout:Get:33 http://security.ubuntu.com/ubuntu focal-security/restricted Translation-en [441 kB] 2024-09-18T10:21:15.140 INFO:teuthology.orchestra.run.smithi134.stdout:Get:34 http://security.ubuntu.com/ubuntu focal-security/restricted amd64 c-n-f Metadata [548 B] 2024-09-18T10:21:15.140 INFO:teuthology.orchestra.run.smithi134.stdout:Get:35 http://security.ubuntu.com/ubuntu focal-security/universe amd64 Packages [1,012 kB] 2024-09-18T10:21:15.167 INFO:teuthology.orchestra.run.smithi134.stdout:Get:36 http://security.ubuntu.com/ubuntu focal-security/universe i386 Packages [681 kB] 2024-09-18T10:21:15.190 INFO:teuthology.orchestra.run.smithi134.stdout:Get:37 http://security.ubuntu.com/ubuntu focal-security/universe Translation-en [213 kB] 2024-09-18T10:21:15.197 INFO:teuthology.orchestra.run.smithi134.stdout:Get:38 http://security.ubuntu.com/ubuntu focal-security/universe amd64 c-n-f Metadata [21.4 kB] 2024-09-18T10:21:15.198 INFO:teuthology.orchestra.run.smithi134.stdout:Get:39 http://security.ubuntu.com/ubuntu focal-security/multiverse i386 Packages [7,204 B] 2024-09-18T10:21:15.198 INFO:teuthology.orchestra.run.smithi134.stdout:Get:40 http://security.ubuntu.com/ubuntu focal-security/multiverse amd64 Packages [24.8 kB] 2024-09-18T10:21:15.270 INFO:teuthology.orchestra.run.smithi134.stdout:Get:41 http://security.ubuntu.com/ubuntu focal-security/multiverse Translation-en [5,968 B] 2024-09-18T10:21:15.270 INFO:teuthology.orchestra.run.smithi134.stdout:Get:42 http://security.ubuntu.com/ubuntu focal-security/multiverse amd64 c-n-f Metadata [540 B] 2024-09-18T10:21:18.732 INFO:teuthology.orchestra.run.smithi134.stdout:Fetched 22.0 MB in 3s (7,015 kB/s) 2024-09-18T10:21:20.227 INFO:teuthology.orchestra.run.smithi134.stdout:Reading package lists... 2024-09-18T10:21:20.255 DEBUG:teuthology.orchestra.run.smithi134:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-09-18T10:21:20.325 INFO:teuthology.orchestra.run.smithi134.stdout:Reading package lists... 2024-09-18T10:21:20.543 INFO:teuthology.orchestra.run.smithi134.stdout:Building dependency tree... 2024-09-18T10:21:20.544 INFO:teuthology.orchestra.run.smithi134.stdout:Reading state information... 2024-09-18T10:21:20.754 INFO:teuthology.orchestra.run.smithi134.stdout:The following additional packages will be installed: 2024-09-18T10:21:20.754 INFO:teuthology.orchestra.run.smithi134.stdout: linux-generic linux-headers-5.4.0-196 linux-headers-5.4.0-196-generic 2024-09-18T10:21:20.754 INFO:teuthology.orchestra.run.smithi134.stdout: linux-headers-generic linux-image-5.4.0-196-generic 2024-09-18T10:21:20.754 INFO:teuthology.orchestra.run.smithi134.stdout: linux-modules-5.4.0-196-generic linux-modules-extra-5.4.0-196-generic 2024-09-18T10:21:20.755 INFO:teuthology.orchestra.run.smithi134.stdout:Suggested packages: 2024-09-18T10:21:20.755 INFO:teuthology.orchestra.run.smithi134.stdout: fdutils linux-doc | linux-source-5.4.0 linux-tools 2024-09-18T10:21:20.755 INFO:teuthology.orchestra.run.smithi134.stdout:Recommended packages: 2024-09-18T10:21:20.755 INFO:teuthology.orchestra.run.smithi134.stdout: thermald 2024-09-18T10:21:20.811 INFO:teuthology.orchestra.run.smithi134.stdout:The following NEW packages will be installed: 2024-09-18T10:21:20.811 INFO:teuthology.orchestra.run.smithi134.stdout: linux-headers-5.4.0-196 linux-headers-5.4.0-196-generic 2024-09-18T10:21:20.811 INFO:teuthology.orchestra.run.smithi134.stdout: linux-image-5.4.0-196-generic linux-modules-5.4.0-196-generic 2024-09-18T10:21:20.811 INFO:teuthology.orchestra.run.smithi134.stdout: linux-modules-extra-5.4.0-196-generic 2024-09-18T10:21:20.813 INFO:teuthology.orchestra.run.smithi134.stdout:The following packages will be upgraded: 2024-09-18T10:21:20.814 INFO:teuthology.orchestra.run.smithi134.stdout: linux-generic linux-headers-generic linux-image-generic 2024-09-18T10:21:20.867 INFO:teuthology.orchestra.run.smithi134.stdout:3 upgraded, 5 newly installed, 0 to remove and 316 not upgraded. 2024-09-18T10:21:20.868 INFO:teuthology.orchestra.run.smithi134.stdout:Need to get 77.2 MB of archives. 2024-09-18T10:21:20.868 INFO:teuthology.orchestra.run.smithi134.stdout:After this operation, 381 MB of additional disk space will be used. 2024-09-18T10:21:20.868 INFO:teuthology.orchestra.run.smithi134.stdout:Get:1 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-modules-5.4.0-196-generic amd64 5.4.0-196.216 [15.0 MB] 2024-09-18T10:21:21.210 INFO:teuthology.orchestra.run.smithi134.stdout:Get:2 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-image-5.4.0-196-generic amd64 5.4.0-196.216 [10.5 MB] 2024-09-18T10:21:21.305 INFO:teuthology.orchestra.run.smithi134.stdout:Get:3 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-modules-extra-5.4.0-196-generic amd64 5.4.0-196.216 [39.2 MB] 2024-09-18T10:21:21.780 INFO:teuthology.orchestra.run.smithi134.stdout:Get:4 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-generic amd64 5.4.0.196.194 [1,900 B] 2024-09-18T10:21:21.780 INFO:teuthology.orchestra.run.smithi134.stdout:Get:5 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-image-generic amd64 5.4.0.196.194 [2,592 B] 2024-09-18T10:21:21.780 INFO:teuthology.orchestra.run.smithi134.stdout:Get:6 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-headers-5.4.0-196 all 5.4.0-196.216 [11.0 MB] 2024-09-18T10:21:21.872 INFO:teuthology.orchestra.run.smithi134.stdout:Get:7 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-headers-5.4.0-196-generic amd64 5.4.0-196.216 [1,386 kB] 2024-09-18T10:21:21.882 INFO:teuthology.orchestra.run.smithi134.stdout:Get:8 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 linux-headers-generic amd64 5.4.0.196.194 [2,460 B] 2024-09-18T10:21:22.603 INFO:teuthology.orchestra.run.smithi134.stdout:Fetched 77.2 MB in 1s (72.7 MB/s) 2024-09-18T10:21:22.801 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package linux-modules-5.4.0-196-generic. 2024-09-18T10:21:24.549 INFO:teuthology.orchestra.run.smithi134.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 104373 files and directories currently installed.) 2024-09-18T10:21:24.554 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../0-linux-modules-5.4.0-196-generic_5.4.0-196.216_amd64.deb ... 2024-09-18T10:21:24.673 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking linux-modules-5.4.0-196-generic (5.4.0-196.216) ... 2024-09-18T10:21:26.707 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package linux-image-5.4.0-196-generic. 2024-09-18T10:21:26.720 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../1-linux-image-5.4.0-196-generic_5.4.0-196.216_amd64.deb ... 2024-09-18T10:21:26.822 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking linux-image-5.4.0-196-generic (5.4.0-196.216) ... 2024-09-18T10:21:27.927 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package linux-modules-extra-5.4.0-196-generic. 2024-09-18T10:21:27.942 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../2-linux-modules-extra-5.4.0-196-generic_5.4.0-196.216_amd64.deb ... 2024-09-18T10:21:27.985 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking linux-modules-extra-5.4.0-196-generic (5.4.0-196.216) ... 2024-09-18T10:21:32.982 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../3-linux-generic_5.4.0.196.194_amd64.deb ... 2024-09-18T10:21:33.100 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking linux-generic (5.4.0.196.194) over (5.4.0.124.125) ... 2024-09-18T10:21:33.452 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../4-linux-image-generic_5.4.0.196.194_amd64.deb ... 2024-09-18T10:21:33.603 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking linux-image-generic (5.4.0.196.194) over (5.4.0.124.125) ... 2024-09-18T10:21:33.948 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package linux-headers-5.4.0-196. 2024-09-18T10:21:33.961 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../5-linux-headers-5.4.0-196_5.4.0-196.216_all.deb ... 2024-09-18T10:21:34.006 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking linux-headers-5.4.0-196 (5.4.0-196.216) ... 2024-09-18T10:21:39.605 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package linux-headers-5.4.0-196-generic. 2024-09-18T10:21:39.626 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../6-linux-headers-5.4.0-196-generic_5.4.0-196.216_amd64.deb ... 2024-09-18T10:21:39.655 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking linux-headers-5.4.0-196-generic (5.4.0-196.216) ... 2024-09-18T10:21:42.392 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../7-linux-headers-generic_5.4.0.196.194_amd64.deb ... 2024-09-18T10:21:42.520 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking linux-headers-generic (5.4.0.196.194) over (5.4.0.124.125) ... 2024-09-18T10:21:42.874 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up linux-headers-5.4.0-196 (5.4.0-196.216) ... 2024-09-18T10:21:42.999 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up linux-headers-5.4.0-196-generic (5.4.0-196.216) ... 2024-09-18T10:21:43.125 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up linux-headers-generic (5.4.0.196.194) ... 2024-09-18T10:21:43.268 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up linux-modules-5.4.0-196-generic (5.4.0-196.216) ... 2024-09-18T10:21:44.487 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up linux-image-5.4.0-196-generic (5.4.0-196.216) ... 2024-09-18T10:21:45.685 INFO:teuthology.orchestra.run.smithi134.stdout:I: /vmlinuz is now a symlink to boot/vmlinuz-5.4.0-196-generic 2024-09-18T10:21:45.685 INFO:teuthology.orchestra.run.smithi134.stdout:I: /initrd.img is now a symlink to boot/initrd.img-5.4.0-196-generic 2024-09-18T10:21:45.839 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up linux-modules-extra-5.4.0-196-generic (5.4.0-196.216) ... 2024-09-18T10:21:47.024 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up linux-image-generic (5.4.0.196.194) ... 2024-09-18T10:21:47.183 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up linux-generic (5.4.0.196.194) ... 2024-09-18T10:21:47.310 INFO:teuthology.orchestra.run.smithi134.stdout:Processing triggers for linux-image-5.4.0-196-generic (5.4.0-196.216) ... 2024-09-18T10:21:47.387 INFO:teuthology.orchestra.run.smithi134.stdout:/etc/kernel/postinst.d/initramfs-tools: 2024-09-18T10:21:47.387 INFO:teuthology.orchestra.run.smithi134.stdout:update-initramfs: Generating /boot/initrd.img-5.4.0-196-generic 2024-09-18T10:22:10.248 INFO:teuthology.orchestra.run.smithi134.stdout:/etc/kernel/postinst.d/zz-update-grub: 2024-09-18T10:22:10.248 INFO:teuthology.orchestra.run.smithi134.stdout:Sourcing file `/etc/default/grub' 2024-09-18T10:22:10.254 INFO:teuthology.orchestra.run.smithi134.stdout:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-09-18T10:22:10.264 INFO:teuthology.orchestra.run.smithi134.stdout:Generating grub configuration file ... 2024-09-18T10:22:10.633 INFO:teuthology.orchestra.run.smithi134.stdout:Found linux image: /boot/vmlinuz-5.4.0-196-generic 2024-09-18T10:22:10.657 INFO:teuthology.orchestra.run.smithi134.stdout:Found initrd image: /boot/initrd.img-5.4.0-196-generic 2024-09-18T10:22:11.139 INFO:teuthology.orchestra.run.smithi134.stdout:Found linux image: /boot/vmlinuz-5.4.0-124-generic 2024-09-18T10:22:11.149 INFO:teuthology.orchestra.run.smithi134.stdout:Found initrd image: /boot/initrd.img-5.4.0-124-generic 2024-09-18T10:22:13.035 INFO:teuthology.orchestra.run.smithi134.stdout:done 2024-09-18T10:22:13.518 DEBUG:teuthology.orchestra.run.smithi134:> dpkg -s linux-image-generic 2024-09-18T10:22:13.573 INFO:teuthology.orchestra.run.smithi134.stdout:Package: linux-image-generic 2024-09-18T10:22:13.573 INFO:teuthology.orchestra.run.smithi134.stdout:Status: install ok installed 2024-09-18T10:22:13.573 INFO:teuthology.orchestra.run.smithi134.stdout:Priority: optional 2024-09-18T10:22:13.574 INFO:teuthology.orchestra.run.smithi134.stdout:Section: kernel 2024-09-18T10:22:13.574 INFO:teuthology.orchestra.run.smithi134.stdout:Installed-Size: 20 2024-09-18T10:22:13.574 INFO:teuthology.orchestra.run.smithi134.stdout:Maintainer: Ubuntu Kernel Team 2024-09-18T10:22:13.574 INFO:teuthology.orchestra.run.smithi134.stdout:Architecture: amd64 2024-09-18T10:22:13.574 INFO:teuthology.orchestra.run.smithi134.stdout:Source: linux-meta 2024-09-18T10:22:13.574 INFO:teuthology.orchestra.run.smithi134.stdout:Version: 5.4.0.196.194 2024-09-18T10:22:13.574 INFO:teuthology.orchestra.run.smithi134.stdout:Provides: virtualbox-guest-modules (= 6.1.50-dfsg-1~ubuntu1.20.04.1), wireguard-modules (= 1.0.20201112-1~20.04.1), zfs-modules (= 0.8.3-1ubuntu12.17) 2024-09-18T10:22:13.574 INFO:teuthology.orchestra.run.smithi134.stdout:Depends: linux-image-5.4.0-196-generic, linux-modules-extra-5.4.0-196-generic, linux-firmware, intel-microcode, amd64-microcode 2024-09-18T10:22:13.574 INFO:teuthology.orchestra.run.smithi134.stdout:Recommends: thermald 2024-09-18T10:22:13.574 INFO:teuthology.orchestra.run.smithi134.stdout:Description: Generic Linux kernel image 2024-09-18T10:22:13.574 INFO:teuthology.orchestra.run.smithi134.stdout: This package will always depend on the latest generic kernel image 2024-09-18T10:22:13.574 INFO:teuthology.orchestra.run.smithi134.stdout: available. 2024-09-18T10:22:13.575 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.4.0-124-generic Expected: 5.4.0-196-generic 2024-09-18T10:22:13.575 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-18T10:22:13.575 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-18T10:22:13.575 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2024-09-18T10:22:13.575 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi134.front.sepia.ceph.com, path=None, version=distro) 2024-09-18T10:22:13.575 DEBUG:teuthology.orchestra.run.smithi134:> sudo apt-get clean 2024-09-18T10:22:13.670 DEBUG:teuthology.orchestra.run.smithi134:> sudo apt-get update 2024-09-18T10:22:13.822 INFO:teuthology.orchestra.run.smithi134.stdout:Hit:1 http://security.ubuntu.com/ubuntu focal-security InRelease 2024-09-18T10:22:13.940 INFO:teuthology.orchestra.run.smithi134.stdout:Hit:2 http://archive.ubuntu.com/ubuntu focal InRelease 2024-09-18T10:22:14.021 INFO:teuthology.orchestra.run.smithi134.stdout:Hit:3 http://archive.ubuntu.com/ubuntu focal-updates InRelease 2024-09-18T10:22:14.103 INFO:teuthology.orchestra.run.smithi134.stdout:Hit:4 http://archive.ubuntu.com/ubuntu focal-backports InRelease 2024-09-18T10:22:15.701 INFO:teuthology.orchestra.run.smithi134.stdout:Reading package lists... 2024-09-18T10:22:15.727 DEBUG:teuthology.orchestra.run.smithi134:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-09-18T10:22:15.799 INFO:teuthology.orchestra.run.smithi134.stdout:Reading package lists... 2024-09-18T10:22:16.013 INFO:teuthology.orchestra.run.smithi134.stdout:Building dependency tree... 2024-09-18T10:22:16.013 INFO:teuthology.orchestra.run.smithi134.stdout:Reading state information... 2024-09-18T10:22:16.249 INFO:teuthology.orchestra.run.smithi134.stdout:linux-image-generic is already the newest version (5.4.0.196.194). 2024-09-18T10:22:16.249 INFO:teuthology.orchestra.run.smithi134.stdout:0 upgraded, 0 newly installed, 0 to remove and 316 not upgraded. 2024-09-18T10:22:16.251 DEBUG:teuthology.orchestra.run.smithi134:> dpkg -s linux-image-generic 2024-09-18T10:22:16.271 INFO:teuthology.orchestra.run.smithi134.stdout:Package: linux-image-generic 2024-09-18T10:22:16.271 INFO:teuthology.orchestra.run.smithi134.stdout:Status: install ok installed 2024-09-18T10:22:16.271 INFO:teuthology.orchestra.run.smithi134.stdout:Priority: optional 2024-09-18T10:22:16.271 INFO:teuthology.orchestra.run.smithi134.stdout:Section: kernel 2024-09-18T10:22:16.271 INFO:teuthology.orchestra.run.smithi134.stdout:Installed-Size: 20 2024-09-18T10:22:16.272 INFO:teuthology.orchestra.run.smithi134.stdout:Maintainer: Ubuntu Kernel Team 2024-09-18T10:22:16.272 INFO:teuthology.orchestra.run.smithi134.stdout:Architecture: amd64 2024-09-18T10:22:16.272 INFO:teuthology.orchestra.run.smithi134.stdout:Source: linux-meta 2024-09-18T10:22:16.272 INFO:teuthology.orchestra.run.smithi134.stdout:Version: 5.4.0.196.194 2024-09-18T10:22:16.272 INFO:teuthology.orchestra.run.smithi134.stdout:Provides: virtualbox-guest-modules (= 6.1.50-dfsg-1~ubuntu1.20.04.1), wireguard-modules (= 1.0.20201112-1~20.04.1), zfs-modules (= 0.8.3-1ubuntu12.17) 2024-09-18T10:22:16.272 INFO:teuthology.orchestra.run.smithi134.stdout:Depends: linux-image-5.4.0-196-generic, linux-modules-extra-5.4.0-196-generic, linux-firmware, intel-microcode, amd64-microcode 2024-09-18T10:22:16.272 INFO:teuthology.orchestra.run.smithi134.stdout:Recommends: thermald 2024-09-18T10:22:16.272 INFO:teuthology.orchestra.run.smithi134.stdout:Description: Generic Linux kernel image 2024-09-18T10:22:16.272 INFO:teuthology.orchestra.run.smithi134.stdout: This package will always depend on the latest generic kernel image 2024-09-18T10:22:16.272 INFO:teuthology.orchestra.run.smithi134.stdout: available. 2024-09-18T10:22:16.273 DEBUG:teuthology.orchestra.run.smithi134:> mktemp 2024-09-18T10:22:16.277 INFO:teuthology.orchestra.run.smithi134.stdout:/tmp/tmp.b1MW5m4uz7 2024-09-18T10:22:16.277 DEBUG:teuthology.orchestra.run.smithi134:> sudo cp /boot/grub/grub.cfg /tmp/tmp.b1MW5m4uz7 2024-09-18T10:22:16.334 DEBUG:teuthology.orchestra.run.smithi134:> sudo chmod 0666 /tmp/tmp.b1MW5m4uz7 2024-09-18T10:22:16.371 DEBUG:teuthology.orchestra.remote:smithi134:/tmp/tmp.b1MW5m4uz7 is 9KB 2024-09-18T10:22:16.384 DEBUG:teuthology.orchestra.run.smithi134:> rm -fr /tmp/tmp.b1MW5m4uz7 2024-09-18T10:22:16.391 DEBUG:teuthology.orchestra.run.smithi134:> sudo rm -f -- /etc/grub.d/01_ceph_kernel 2024-09-18T10:22:16.449 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-09-18T10:22:16.449 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd of=/etc/grub.d/01_ceph_kernel 2024-09-18T10:22:16.449 DEBUG:teuthology.orchestra.run.smithi134:> sudo chmod 755 /etc/grub.d/01_ceph_kernel 2024-09-18T10:22:16.487 INFO:teuthology.task.kernel:Distro Kernel Version: 5.4.0-196-generic 2024-09-18T10:22:16.488 DEBUG:teuthology.orchestra.run.smithi134:> sudo update-grub 2024-09-18T10:22:17.952 INFO:teuthology.orchestra.run.smithi134.stderr:Sourcing file `/etc/default/grub' 2024-09-18T10:22:17.957 INFO:teuthology.orchestra.run.smithi134.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-09-18T10:22:17.965 INFO:teuthology.orchestra.run.smithi134.stderr:Generating grub configuration file ... 2024-09-18T10:22:18.264 INFO:teuthology.orchestra.run.smithi134.stderr:Found linux image: /boot/vmlinuz-5.4.0-196-generic 2024-09-18T10:22:18.276 INFO:teuthology.orchestra.run.smithi134.stderr:Found initrd image: /boot/initrd.img-5.4.0-196-generic 2024-09-18T10:22:18.723 INFO:teuthology.orchestra.run.smithi134.stderr:Found linux image: /boot/vmlinuz-5.4.0-124-generic 2024-09-18T10:22:18.735 INFO:teuthology.orchestra.run.smithi134.stderr:Found initrd image: /boot/initrd.img-5.4.0-124-generic 2024-09-18T10:22:19.501 INFO:teuthology.orchestra.run.smithi134.stderr:done 2024-09-18T10:22:19.502 DEBUG:teuthology.orchestra.run.smithi134:> sudo shutdown -r now 2024-09-18T10:22:49.530 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-09-18T10:22:49.530 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi134.front.sepia.ceph.com' 2024-09-18T10:22:49.531 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi134.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-18T10:23:49.593 DEBUG:teuthology.orchestra.remote:timed out 2024-09-18T10:23:58.600 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi134.front.sepia.ceph.com' 2024-09-18T10:23:58.601 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi134.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-18T10:23:58.989 DEBUG:teuthology.orchestra.run.smithi134:> true 2024-09-18T10:23:59.920 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi134.front.sepia.ceph.com' 2024-09-18T10:23:59.920 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.4.0-196-generic"... 2024-09-18T10:23:59.921 DEBUG:teuthology.orchestra.run.smithi134:> uname -r 2024-09-18T10:23:59.927 INFO:teuthology.orchestra.run.smithi134.stdout:5.4.0-196-generic 2024-09-18T10:23:59.928 DEBUG:teuthology.task.kernel:current kernel version is 5.4.0-196-generic vs 5.4.0-196-generic 2024-09-18T10:23:59.928 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-18T10:23:59.928 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2024-09-18T10:24:00.928 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-09-18T10:24:00.929 DEBUG:teuthology.orchestra.run.smithi134:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-18T10:24:01.027 INFO:teuthology.orchestra.run.smithi134.stdout:ttyS1 2024-09-18T10:24:01.046 DEBUG:teuthology.parallel:result is None 2024-09-18T10:24:01.046 INFO:teuthology.run_tasks:Running task internal.base... 2024-09-18T10:24:01.053 INFO:teuthology.task.internal:Creating test directory... 2024-09-18T10:24:01.053 DEBUG:teuthology.orchestra.run.smithi134:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-18T10:24:01.060 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-09-18T10:24:01.066 INFO:teuthology.run_tasks:Running task internal.archive... 2024-09-18T10:24:01.076 INFO:teuthology.task.internal:Creating archive directory... 2024-09-18T10:24:01.076 DEBUG:teuthology.orchestra.run.smithi134:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-18T10:24:01.097 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-09-18T10:24:01.103 INFO:teuthology.task.internal:Enabling coredump saving... 2024-09-18T10:24:01.103 DEBUG:teuthology.orchestra.run.smithi134:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-18T10:24:01.166 INFO:teuthology.orchestra.run.smithi134.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-18T10:24:01.176 INFO:teuthology.orchestra.run.smithi134.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-18T10:24:01.178 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-09-18T10:24:01.186 INFO:teuthology.task.internal:Configuring sudo... 2024-09-18T10:24:01.186 DEBUG:teuthology.orchestra.run.smithi134:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-18T10:24:01.202 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-09-18T10:24:01.210 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-09-18T10:24:01.210 DEBUG:teuthology.orchestra.run.smithi134:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-18T10:24:01.216 DEBUG:teuthology.orchestra.run.smithi134:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-18T10:24:01.270 DEBUG:teuthology.orchestra.run.smithi134:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-18T10:24:01.276 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-09-18T10:24:01.277 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-18T10:24:01.302 DEBUG:teuthology.orchestra.run.smithi134:> sudo service rsyslog restart 2024-09-18T10:24:01.395 INFO:teuthology.run_tasks:Running task internal.timer... 2024-09-18T10:24:01.403 INFO:teuthology.task.internal:Starting timer... 2024-09-18T10:24:01.403 INFO:teuthology.run_tasks:Running task pcp... 2024-09-18T10:24:01.413 INFO:teuthology.run_tasks:Running task selinux... 2024-09-18T10:24:01.422 DEBUG:teuthology.task.selinux:Excluding smithi134: OS 'ubuntu' does not support SELinux 2024-09-18T10:24:01.422 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-09-18T10:24:01.422 DEBUG:teuthology.task.selinux:Existing SELinux modes: {} 2024-09-18T10:24:01.422 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-09-18T10:24:01.422 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-09-18T10:24:01.430 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-09-18T10:24:01.442 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-09-18T10:24:01.443 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi134.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-09-18T10:29:38.123 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi134.front.sepia.ceph.com')] 2024-09-18T10:29:38.124 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi134.front.sepia.ceph.com' 2024-09-18T10:29:38.125 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi134.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-18T10:29:38.231 DEBUG:teuthology.orchestra.run.smithi134:> true 2024-09-18T10:29:38.350 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi134.front.sepia.ceph.com' 2024-09-18T10:29:38.351 INFO:teuthology.run_tasks:Running task clock... 2024-09-18T10:29:38.368 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-09-18T10:29:38.368 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-18T10:29:38.369 DEBUG:teuthology.orchestra.run.smithi134:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-18T10:29:38.405 INFO:teuthology.orchestra.run.smithi134.stdout:18 Sep 10:29:38 ntpd[13736]: ntpd 4.2.8p12@1.3728-o (1): Starting 2024-09-18T10:29:38.405 INFO:teuthology.orchestra.run.smithi134.stdout:18 Sep 10:29:38 ntpd[13736]: Command line: ntpd -gq 2024-09-18T10:29:38.406 INFO:teuthology.orchestra.run.smithi134.stdout:18 Sep 10:29:38 ntpd[13736]: proto: precision = 0.093 usec (-23) 2024-09-18T10:29:38.407 INFO:teuthology.orchestra.run.smithi134.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-09-18T10:29:38.407 INFO:teuthology.orchestra.run.smithi134.stdout:18 Sep 10:29:38 ntpd[13736]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-09-18T10:29:38.408 INFO:teuthology.orchestra.run.smithi134.stderr:restrict ::: KOD does nothing without LIMITED. 2024-09-18T10:29:38.410 INFO:teuthology.orchestra.run.smithi134.stdout:18 Sep 10:29:38 ntpd[13736]: restrict ::: KOD does nothing without LIMITED. 2024-09-18T10:29:38.410 INFO:teuthology.orchestra.run.smithi134.stdout:18 Sep 10:29:38 ntpd[13736]: Listen and drop on 0 v6wildcard [::]:123 2024-09-18T10:29:38.410 INFO:teuthology.orchestra.run.smithi134.stdout:18 Sep 10:29:38 ntpd[13736]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2024-09-18T10:29:38.410 INFO:teuthology.orchestra.run.smithi134.stdout:18 Sep 10:29:38 ntpd[13736]: Listen normally on 2 lo 127.0.0.1:123 2024-09-18T10:29:38.410 INFO:teuthology.orchestra.run.smithi134.stdout:18 Sep 10:29:38 ntpd[13736]: Listen normally on 3 enp3s0f1 172.21.15.134:123 2024-09-18T10:29:38.410 INFO:teuthology.orchestra.run.smithi134.stdout:18 Sep 10:29:38 ntpd[13736]: Listen normally on 4 lo [::1]:123 2024-09-18T10:29:38.411 INFO:teuthology.orchestra.run.smithi134.stdout:18 Sep 10:29:38 ntpd[13736]: Listen normally on 5 enp3s0f1 [fe80::ec4:7aff:fe88:6de9%5]:123 2024-09-18T10:29:38.411 INFO:teuthology.orchestra.run.smithi134.stdout:18 Sep 10:29:38 ntpd[13736]: Listening on routing socket on fd #22 for interface updates 2024-09-18T10:29:39.406 INFO:teuthology.orchestra.run.smithi134.stderr:18 Sep 10:29:39 ntpd[13736]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-18T10:29:39.406 INFO:teuthology.orchestra.run.smithi134.stderr:18 Sep 10:29:39 ntpd[13736]: can't open /var/log/ntpstats/rawstats.20240918: Permission denied 2024-09-18T10:29:39.406 INFO:teuthology.orchestra.run.smithi134.stderr:18 Sep 10:29:39 ntpd[13736]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-18T10:29:39.406 INFO:teuthology.orchestra.run.smithi134.stderr:18 Sep 10:29:39 ntpd[13736]: can't open /var/log/ntpstats/rawstats.20240918: Permission denied 2024-09-18T10:29:39.406 INFO:teuthology.orchestra.run.smithi134.stderr:18 Sep 10:29:39 ntpd[13736]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-18T10:29:39.406 INFO:teuthology.orchestra.run.smithi134.stderr:18 Sep 10:29:39 ntpd[13736]: can't open /var/log/ntpstats/rawstats.20240918: Permission denied 2024-09-18T10:29:39.406 INFO:teuthology.orchestra.run.smithi134.stderr:18 Sep 10:29:39 ntpd[13736]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-18T10:29:39.406 INFO:teuthology.orchestra.run.smithi134.stderr:18 Sep 10:29:39 ntpd[13736]: can't open /var/log/ntpstats/peerstats.20240918: Permission denied 2024-09-18T10:29:41.406 INFO:teuthology.orchestra.run.smithi134.stderr:18 Sep 10:29:41 ntpd[13736]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-18T10:29:41.406 INFO:teuthology.orchestra.run.smithi134.stderr:18 Sep 10:29:41 ntpd[13736]: can't open /var/log/ntpstats/rawstats.20240918: Permission denied 2024-09-18T10:29:41.407 INFO:teuthology.orchestra.run.smithi134.stderr:18 Sep 10:29:41 ntpd[13736]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-18T10:29:41.407 INFO:teuthology.orchestra.run.smithi134.stderr:18 Sep 10:29:41 ntpd[13736]: can't open /var/log/ntpstats/rawstats.20240918: Permission denied 2024-09-18T10:29:41.407 INFO:teuthology.orchestra.run.smithi134.stderr:18 Sep 10:29:41 ntpd[13736]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-18T10:29:41.407 INFO:teuthology.orchestra.run.smithi134.stderr:18 Sep 10:29:41 ntpd[13736]: can't open /var/log/ntpstats/rawstats.20240918: Permission denied 2024-09-18T10:29:41.407 INFO:teuthology.orchestra.run.smithi134.stderr:18 Sep 10:29:41 ntpd[13736]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-18T10:29:41.407 INFO:teuthology.orchestra.run.smithi134.stderr:18 Sep 10:29:41 ntpd[13736]: can't open /var/log/ntpstats/peerstats.20240918: Permission denied 2024-09-18T10:29:41.407 INFO:teuthology.orchestra.run.smithi134.stderr:18 Sep 10:29:41 ntpd[13736]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-18T10:29:41.407 INFO:teuthology.orchestra.run.smithi134.stderr:18 Sep 10:29:41 ntpd[13736]: can't open /var/log/ntpstats/rawstats.20240918: Permission denied 2024-09-18T10:29:41.407 INFO:teuthology.orchestra.run.smithi134.stderr:18 Sep 10:29:41 ntpd[13736]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-18T10:29:41.407 INFO:teuthology.orchestra.run.smithi134.stderr:18 Sep 10:29:41 ntpd[13736]: can't open /var/log/ntpstats/peerstats.20240918: Permission denied 2024-09-18T10:29:42.406 INFO:teuthology.orchestra.run.smithi134.stderr:18 Sep 10:29:42 ntpd[13736]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-18T10:29:42.406 INFO:teuthology.orchestra.run.smithi134.stderr:18 Sep 10:29:42 ntpd[13736]: can't open /var/log/ntpstats/rawstats.20240918: Permission denied 2024-09-18T10:29:42.406 INFO:teuthology.orchestra.run.smithi134.stderr:18 Sep 10:29:42 ntpd[13736]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-18T10:29:42.406 INFO:teuthology.orchestra.run.smithi134.stderr:18 Sep 10:29:42 ntpd[13736]: can't open /var/log/ntpstats/rawstats.20240918: Permission denied 2024-09-18T10:29:42.406 INFO:teuthology.orchestra.run.smithi134.stderr:18 Sep 10:29:42 ntpd[13736]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-18T10:29:42.406 INFO:teuthology.orchestra.run.smithi134.stderr:18 Sep 10:29:42 ntpd[13736]: can't open /var/log/ntpstats/peerstats.20240918: Permission denied 2024-09-18T10:29:43.406 INFO:teuthology.orchestra.run.smithi134.stderr:18 Sep 10:29:43 ntpd[13736]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-18T10:29:43.407 INFO:teuthology.orchestra.run.smithi134.stderr:18 Sep 10:29:43 ntpd[13736]: can't open /var/log/ntpstats/rawstats.20240918: Permission denied 2024-09-18T10:29:43.407 INFO:teuthology.orchestra.run.smithi134.stderr:18 Sep 10:29:43 ntpd[13736]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-18T10:29:43.407 INFO:teuthology.orchestra.run.smithi134.stderr:18 Sep 10:29:43 ntpd[13736]: can't open /var/log/ntpstats/rawstats.20240918: Permission denied 2024-09-18T10:29:43.407 INFO:teuthology.orchestra.run.smithi134.stderr:18 Sep 10:29:43 ntpd[13736]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-18T10:29:43.407 INFO:teuthology.orchestra.run.smithi134.stderr:18 Sep 10:29:43 ntpd[13736]: can't open /var/log/ntpstats/rawstats.20240918: Permission denied 2024-09-18T10:29:43.407 INFO:teuthology.orchestra.run.smithi134.stderr:18 Sep 10:29:43 ntpd[13736]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-18T10:29:43.407 INFO:teuthology.orchestra.run.smithi134.stderr:18 Sep 10:29:43 ntpd[13736]: can't open /var/log/ntpstats/peerstats.20240918: Permission denied 2024-09-18T10:29:43.407 INFO:teuthology.orchestra.run.smithi134.stderr:18 Sep 10:29:43 ntpd[13736]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-18T10:29:43.407 INFO:teuthology.orchestra.run.smithi134.stderr:18 Sep 10:29:43 ntpd[13736]: can't open /var/log/ntpstats/rawstats.20240918: Permission denied 2024-09-18T10:29:43.407 INFO:teuthology.orchestra.run.smithi134.stderr:18 Sep 10:29:43 ntpd[13736]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-18T10:29:43.407 INFO:teuthology.orchestra.run.smithi134.stderr:18 Sep 10:29:43 ntpd[13736]: can't open /var/log/ntpstats/peerstats.20240918: Permission denied 2024-09-18T10:29:44.406 INFO:teuthology.orchestra.run.smithi134.stderr:18 Sep 10:29:44 ntpd[13736]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-18T10:29:44.406 INFO:teuthology.orchestra.run.smithi134.stderr:18 Sep 10:29:44 ntpd[13736]: can't open /var/log/ntpstats/rawstats.20240918: Permission denied 2024-09-18T10:29:44.406 INFO:teuthology.orchestra.run.smithi134.stderr:18 Sep 10:29:44 ntpd[13736]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-18T10:29:44.406 INFO:teuthology.orchestra.run.smithi134.stderr:18 Sep 10:29:44 ntpd[13736]: can't open /var/log/ntpstats/rawstats.20240918: Permission denied 2024-09-18T10:29:44.406 INFO:teuthology.orchestra.run.smithi134.stderr:18 Sep 10:29:44 ntpd[13736]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-18T10:29:44.406 INFO:teuthology.orchestra.run.smithi134.stderr:18 Sep 10:29:44 ntpd[13736]: can't open /var/log/ntpstats/peerstats.20240918: Permission denied 2024-09-18T10:29:45.406 INFO:teuthology.orchestra.run.smithi134.stderr:18 Sep 10:29:45 ntpd[13736]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-18T10:29:45.406 INFO:teuthology.orchestra.run.smithi134.stderr:18 Sep 10:29:45 ntpd[13736]: can't open /var/log/ntpstats/rawstats.20240918: Permission denied 2024-09-18T10:29:45.407 INFO:teuthology.orchestra.run.smithi134.stderr:18 Sep 10:29:45 ntpd[13736]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-18T10:29:45.407 INFO:teuthology.orchestra.run.smithi134.stderr:18 Sep 10:29:45 ntpd[13736]: can't open /var/log/ntpstats/rawstats.20240918: Permission denied 2024-09-18T10:29:45.407 INFO:teuthology.orchestra.run.smithi134.stderr:18 Sep 10:29:45 ntpd[13736]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-09-18T10:29:45.407 INFO:teuthology.orchestra.run.smithi134.stderr:18 Sep 10:29:45 ntpd[13736]: can't open /var/log/ntpstats/rawstats.20240918: Permission denied 2024-09-18T10:29:45.407 INFO:teuthology.orchestra.run.smithi134.stderr:18 Sep 10:29:45 ntpd[13736]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-09-18T10:29:45.407 INFO:teuthology.orchestra.run.smithi134.stderr:18 Sep 10:29:45 ntpd[13736]: can't open /var/log/ntpstats/peerstats.20240918: Permission denied 2024-09-18T10:29:45.407 INFO:teuthology.orchestra.run.smithi134.stdout:18 Sep 10:29:45 ntpd[13736]: ntpd: time slew +0.000817 s 2024-09-18T10:29:45.407 INFO:teuthology.orchestra.run.smithi134.stdout:ntpd: time slew +0.000817s 2024-09-18T10:29:45.408 INFO:teuthology.orchestra.run.smithi134.stderr:18 Sep 10:29:45 ntpd[13736]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2024-09-18T10:29:45.408 INFO:teuthology.orchestra.run.smithi134.stderr:18 Sep 10:29:45 ntpd[13736]: can't open /var/log/ntpstats/loopstats.20240918: Permission denied 2024-09-18T10:29:45.474 INFO:teuthology.orchestra.run.smithi134.stdout: remote refid st t when poll reach delay offset jitter 2024-09-18T10:29:45.474 INFO:teuthology.orchestra.run.smithi134.stdout:============================================================================== 2024-09-18T10:29:45.474 INFO:teuthology.orchestra.run.smithi134.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2024-09-18T10:29:45.474 INFO:teuthology.orchestra.run.smithi134.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2024-09-18T10:29:45.475 INFO:teuthology.orchestra.run.smithi134.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2024-09-18T10:29:45.475 INFO:teuthology.orchestra.run.smithi134.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2024-09-18T10:29:45.476 INFO:teuthology.run_tasks:Running task install... 2024-09-18T10:29:45.484 DEBUG:teuthology.task.install:project ceph 2024-09-18T10:29:45.484 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '7061ea6ca50908d104e9f3979dc615e118d9e153'}} 2024-09-18T10:29:45.484 DEBUG:teuthology.task.install:config {'flavor': 'default', 'sha1': '7061ea6ca50908d104e9f3979dc615e118d9e153'} 2024-09-18T10:29:45.484 INFO:teuthology.task.install:Using flavor: default 2024-09-18T10:29:45.490 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-09-18T10:29:45.490 INFO:teuthology.task.install:extra packages: [] 2024-09-18T10:29:45.490 DEBUG:teuthology.orchestra.run.smithi134:> sudo apt-key list | grep Ceph 2024-09-18T10:29:45.620 INFO:teuthology.orchestra.run.smithi134.stderr:Warning: apt-key output should not be parsed (stdout is not a terminal) 2024-09-18T10:29:45.679 INFO:teuthology.orchestra.run.smithi134.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2024-09-18T10:29:45.679 INFO:teuthology.orchestra.run.smithi134.stdout:uid [ unknown] Ceph.com (release key) 2024-09-18T10:29:45.680 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse on remote deb x86_64 2024-09-18T10:29:45.680 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&sha1=7061ea6ca50908d104e9f3979dc615e118d9e153 2024-09-18T10:29:45.827 INFO:teuthology.task.install.deb:Pulling from https://2.chacra.ceph.com/r/ceph/wip-bharath11-testing-2024-09-18-1115-quincy/7061ea6ca50908d104e9f3979dc615e118d9e153/ubuntu/focal/flavors/default/ 2024-09-18T10:29:45.827 INFO:teuthology.task.install.deb:Package version is 17.2.7-1645-g7061ea6c-1focal 2024-09-18T10:29:45.939 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-09-18T10:29:45.939 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2024-09-18T10:29:45.955 DEBUG:teuthology.orchestra.run.smithi134:> sudo apt-get update 2024-09-18T10:29:46.147 INFO:teuthology.orchestra.run.smithi134.stdout:Ign:1 https://2.chacra.ceph.com/r/ceph/wip-bharath11-testing-2024-09-18-1115-quincy/7061ea6ca50908d104e9f3979dc615e118d9e153/ubuntu/focal/flavors/default focal InRelease 2024-09-18T10:29:46.169 INFO:teuthology.orchestra.run.smithi134.stdout:Get:2 https://2.chacra.ceph.com/r/ceph/wip-bharath11-testing-2024-09-18-1115-quincy/7061ea6ca50908d104e9f3979dc615e118d9e153/ubuntu/focal/flavors/default focal Release [7,662 B] 2024-09-18T10:29:46.192 INFO:teuthology.orchestra.run.smithi134.stdout:Ign:3 https://2.chacra.ceph.com/r/ceph/wip-bharath11-testing-2024-09-18-1115-quincy/7061ea6ca50908d104e9f3979dc615e118d9e153/ubuntu/focal/flavors/default focal Release.gpg 2024-09-18T10:29:46.222 INFO:teuthology.orchestra.run.smithi134.stdout:Hit:4 http://archive.ubuntu.com/ubuntu focal InRelease 2024-09-18T10:29:46.222 INFO:teuthology.orchestra.run.smithi134.stdout:Hit:5 http://security.ubuntu.com/ubuntu focal-security InRelease 2024-09-18T10:29:46.222 INFO:teuthology.orchestra.run.smithi134.stdout:Get:6 https://2.chacra.ceph.com/r/ceph/wip-bharath11-testing-2024-09-18-1115-quincy/7061ea6ca50908d104e9f3979dc615e118d9e153/ubuntu/focal/flavors/default focal/main i386 Packages [3,390 B] 2024-09-18T10:29:46.256 INFO:teuthology.orchestra.run.smithi134.stdout:Get:7 https://2.chacra.ceph.com/r/ceph/wip-bharath11-testing-2024-09-18-1115-quincy/7061ea6ca50908d104e9f3979dc615e118d9e153/ubuntu/focal/flavors/default focal/main amd64 Packages [17.4 kB] 2024-09-18T10:29:46.291 INFO:teuthology.orchestra.run.smithi134.stdout:Hit:8 http://archive.ubuntu.com/ubuntu focal-updates InRelease 2024-09-18T10:29:46.373 INFO:teuthology.orchestra.run.smithi134.stdout:Hit:9 http://archive.ubuntu.com/ubuntu focal-backports InRelease 2024-09-18T10:29:46.890 INFO:teuthology.orchestra.run.smithi134.stdout:Fetched 28.4 kB in 1s (36.6 kB/s) 2024-09-18T10:29:48.057 INFO:teuthology.orchestra.run.smithi134.stdout:Reading package lists... 2024-09-18T10:29:48.082 DEBUG:teuthology.orchestra.run.smithi134:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=17.2.7-1645-g7061ea6c-1focal cephadm=17.2.7-1645-g7061ea6c-1focal ceph-mds=17.2.7-1645-g7061ea6c-1focal ceph-mgr=17.2.7-1645-g7061ea6c-1focal ceph-common=17.2.7-1645-g7061ea6c-1focal ceph-fuse=17.2.7-1645-g7061ea6c-1focal ceph-test=17.2.7-1645-g7061ea6c-1focal ceph-volume=17.2.7-1645-g7061ea6c-1focal radosgw=17.2.7-1645-g7061ea6c-1focal python3-rados=17.2.7-1645-g7061ea6c-1focal python3-rgw=17.2.7-1645-g7061ea6c-1focal python3-cephfs=17.2.7-1645-g7061ea6c-1focal python3-rbd=17.2.7-1645-g7061ea6c-1focal libcephfs2=17.2.7-1645-g7061ea6c-1focal libcephfs-dev=17.2.7-1645-g7061ea6c-1focal librados2=17.2.7-1645-g7061ea6c-1focal librbd1=17.2.7-1645-g7061ea6c-1focal rbd-fuse=17.2.7-1645-g7061ea6c-1focal 2024-09-18T10:29:48.154 INFO:teuthology.orchestra.run.smithi134.stdout:Reading package lists... 2024-09-18T10:29:48.348 INFO:teuthology.orchestra.run.smithi134.stdout:Building dependency tree... 2024-09-18T10:29:48.349 INFO:teuthology.orchestra.run.smithi134.stdout:Reading state information... 2024-09-18T10:29:48.542 INFO:teuthology.orchestra.run.smithi134.stdout:The following packages were automatically installed and are no longer required: 2024-09-18T10:29:48.544 INFO:teuthology.orchestra.run.smithi134.stdout: libboost-iostreams1.71.0 libboost-thread1.71.0 2024-09-18T10:29:48.544 INFO:teuthology.orchestra.run.smithi134.stdout:Use 'sudo apt autoremove' to remove them. 2024-09-18T10:29:48.544 INFO:teuthology.orchestra.run.smithi134.stdout:The following additional packages will be installed: 2024-09-18T10:29:48.544 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2024-09-18T10:29:48.544 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd fonts-mathjax jq 2024-09-18T10:29:48.544 INFO:teuthology.orchestra.run.smithi134.stdout: libdouble-conversion3 libjq1 libjs-mathjax libjs-sphinxdoc libjs-underscore 2024-09-18T10:29:48.544 INFO:teuthology.orchestra.run.smithi134.stdout: liblbfgsb0 liblttng-ust0 liblua5.3-0 liboath0 libonig5 libpcre2-16-0 2024-09-18T10:29:48.544 INFO:teuthology.orchestra.run.smithi134.stdout: libqt5core5a libqt5dbus5 libqt5network5 librabbitmq4 libradosstriper1 2024-09-18T10:29:48.545 INFO:teuthology.orchestra.run.smithi134.stdout: librdkafka1 librgw2 libsqlite3-mod-ceph libthrift-0.13.0 nvme-cli 2024-09-18T10:29:48.545 INFO:teuthology.orchestra.run.smithi134.stdout: python-natsort-doc python-pastedeploy-tpl python3-atomicwrites python3-attr 2024-09-18T10:29:48.545 INFO:teuthology.orchestra.run.smithi134.stdout: python3-bcrypt python3-blinker python3-bs4 python3-cachetools 2024-09-18T10:29:48.545 INFO:teuthology.orchestra.run.smithi134.stdout: python3-ceph-argparse python3-ceph-common python3-cherrypy3 2024-09-18T10:29:48.545 INFO:teuthology.orchestra.run.smithi134.stdout: python3-cryptography python3-decorator python3-google-auth python3-html5lib 2024-09-18T10:29:48.546 INFO:teuthology.orchestra.run.smithi134.stdout: python3-jinja2 python3-joblib python3-jwt python3-kubernetes 2024-09-18T10:29:48.546 INFO:teuthology.orchestra.run.smithi134.stdout: python3-logutils python3-lxml python3-mako python3-markupsafe 2024-09-18T10:29:48.546 INFO:teuthology.orchestra.run.smithi134.stdout: python3-natsort python3-oauthlib python3-openssl python3-packaging 2024-09-18T10:29:48.546 INFO:teuthology.orchestra.run.smithi134.stdout: python3-paste python3-pastedeploy python3-pastescript python3-pecan 2024-09-18T10:29:48.546 INFO:teuthology.orchestra.run.smithi134.stdout: python3-pluggy python3-prettytable python3-psutil python3-py python3-pyasn1 2024-09-18T10:29:48.546 INFO:teuthology.orchestra.run.smithi134.stdout: python3-pyasn1-modules python3-pyinotify python3-pytest python3-repoze.lru 2024-09-18T10:29:48.546 INFO:teuthology.orchestra.run.smithi134.stdout: python3-requests-oauthlib python3-routes python3-rsa python3-scipy 2024-09-18T10:29:48.546 INFO:teuthology.orchestra.run.smithi134.stdout: python3-simplegeneric python3-simplejson python3-singledispatch 2024-09-18T10:29:48.546 INFO:teuthology.orchestra.run.smithi134.stdout: python3-sklearn python3-sklearn-lib python3-soupsieve python3-tempita 2024-09-18T10:29:48.546 INFO:teuthology.orchestra.run.smithi134.stdout: python3-waitress python3-wcwidth python3-webencodings python3-webob 2024-09-18T10:29:48.546 INFO:teuthology.orchestra.run.smithi134.stdout: python3-websocket python3-webtest python3-werkzeug qttranslations5-l10n 2024-09-18T10:29:48.546 INFO:teuthology.orchestra.run.smithi134.stdout: socat xmlstarlet 2024-09-18T10:29:48.547 INFO:teuthology.orchestra.run.smithi134.stdout:Suggested packages: 2024-09-18T10:29:48.547 INFO:teuthology.orchestra.run.smithi134.stdout: python3-influxdb fonts-mathjax-extras fonts-stix libjs-mathjax-doc 2024-09-18T10:29:48.547 INFO:teuthology.orchestra.run.smithi134.stdout: python-attr-doc python-blinker-doc python-cryptography-doc 2024-09-18T10:29:48.547 INFO:teuthology.orchestra.run.smithi134.stdout: python3-cryptography-vectors python3-genshi python-jinja2-doc python3-crypto 2024-09-18T10:29:48.547 INFO:teuthology.orchestra.run.smithi134.stdout: python3-lxml-dbg python-lxml-doc python3-beaker python-mako-doc 2024-09-18T10:29:48.547 INFO:teuthology.orchestra.run.smithi134.stdout: python-openssl-doc python3-openssl-dbg httpd-wsgi libapache2-mod-python 2024-09-18T10:29:48.547 INFO:teuthology.orchestra.run.smithi134.stdout: libapache2-mod-scgi libjs-mochikit python-pecan-doc python-psutil-doc 2024-09-18T10:29:48.547 INFO:teuthology.orchestra.run.smithi134.stdout: subversion python-pyinotify-doc python-scipy-doc python3-dap 2024-09-18T10:29:48.547 INFO:teuthology.orchestra.run.smithi134.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2024-09-18T10:29:48.547 INFO:teuthology.orchestra.run.smithi134.stdout: python-webtest-doc python-werkzeug-doc python3-termcolor python3-watchdog 2024-09-18T10:29:48.548 INFO:teuthology.orchestra.run.smithi134.stdout:Recommended packages: 2024-09-18T10:29:48.548 INFO:teuthology.orchestra.run.smithi134.stdout: btrfs-tools 2024-09-18T10:29:48.667 INFO:teuthology.orchestra.run.smithi134.stdout:The following NEW packages will be installed: 2024-09-18T10:29:48.667 INFO:teuthology.orchestra.run.smithi134.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2024-09-18T10:29:48.667 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2024-09-18T10:29:48.667 INFO:teuthology.orchestra.run.smithi134.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm 2024-09-18T10:29:48.667 INFO:teuthology.orchestra.run.smithi134.stdout: fonts-mathjax jq libcephfs-dev libcephfs2 libdouble-conversion3 libjq1 2024-09-18T10:29:48.667 INFO:teuthology.orchestra.run.smithi134.stdout: libjs-mathjax libjs-sphinxdoc libjs-underscore liblbfgsb0 liblttng-ust0 2024-09-18T10:29:48.667 INFO:teuthology.orchestra.run.smithi134.stdout: liblua5.3-0 liboath0 libonig5 libpcre2-16-0 libqt5core5a libqt5dbus5 2024-09-18T10:29:48.667 INFO:teuthology.orchestra.run.smithi134.stdout: libqt5network5 librabbitmq4 libradosstriper1 librdkafka1 librgw2 2024-09-18T10:29:48.668 INFO:teuthology.orchestra.run.smithi134.stdout: libsqlite3-mod-ceph libthrift-0.13.0 nvme-cli python-natsort-doc 2024-09-18T10:29:48.668 INFO:teuthology.orchestra.run.smithi134.stdout: python-pastedeploy-tpl python3-atomicwrites python3-attr python3-bcrypt 2024-09-18T10:29:48.668 INFO:teuthology.orchestra.run.smithi134.stdout: python3-blinker python3-bs4 python3-cachetools python3-ceph-argparse 2024-09-18T10:29:48.668 INFO:teuthology.orchestra.run.smithi134.stdout: python3-ceph-common python3-cephfs python3-cherrypy3 python3-cryptography 2024-09-18T10:29:48.668 INFO:teuthology.orchestra.run.smithi134.stdout: python3-decorator python3-google-auth python3-html5lib python3-jinja2 2024-09-18T10:29:48.668 INFO:teuthology.orchestra.run.smithi134.stdout: python3-joblib python3-jwt python3-kubernetes python3-logutils python3-lxml 2024-09-18T10:29:48.669 INFO:teuthology.orchestra.run.smithi134.stdout: python3-mako python3-markupsafe python3-natsort python3-oauthlib 2024-09-18T10:29:48.669 INFO:teuthology.orchestra.run.smithi134.stdout: python3-openssl python3-packaging python3-paste python3-pastedeploy 2024-09-18T10:29:48.669 INFO:teuthology.orchestra.run.smithi134.stdout: python3-pastescript python3-pecan python3-pluggy python3-prettytable 2024-09-18T10:29:48.669 INFO:teuthology.orchestra.run.smithi134.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2024-09-18T10:29:48.669 INFO:teuthology.orchestra.run.smithi134.stdout: python3-pyinotify python3-pytest python3-rados python3-rbd 2024-09-18T10:29:48.669 INFO:teuthology.orchestra.run.smithi134.stdout: python3-repoze.lru python3-requests-oauthlib python3-rgw python3-routes 2024-09-18T10:29:48.669 INFO:teuthology.orchestra.run.smithi134.stdout: python3-rsa python3-scipy python3-simplegeneric python3-simplejson 2024-09-18T10:29:48.669 INFO:teuthology.orchestra.run.smithi134.stdout: python3-singledispatch python3-sklearn python3-sklearn-lib python3-soupsieve 2024-09-18T10:29:48.669 INFO:teuthology.orchestra.run.smithi134.stdout: python3-tempita python3-waitress python3-wcwidth python3-webencodings 2024-09-18T10:29:48.669 INFO:teuthology.orchestra.run.smithi134.stdout: python3-webob python3-websocket python3-webtest python3-werkzeug 2024-09-18T10:29:48.669 INFO:teuthology.orchestra.run.smithi134.stdout: qttranslations5-l10n radosgw rbd-fuse socat xmlstarlet 2024-09-18T10:29:48.670 INFO:teuthology.orchestra.run.smithi134.stdout:The following packages will be upgraded: 2024-09-18T10:29:48.670 INFO:teuthology.orchestra.run.smithi134.stdout: librados2 librbd1 2024-09-18T10:29:48.819 INFO:teuthology.orchestra.run.smithi134.stdout:2 upgraded, 108 newly installed, 0 to remove and 295 not upgraded. 2024-09-18T10:29:48.819 INFO:teuthology.orchestra.run.smithi134.stdout:Need to get 201 MB of archives. 2024-09-18T10:29:48.819 INFO:teuthology.orchestra.run.smithi134.stdout:After this operation, 938 MB of additional disk space will be used. 2024-09-18T10:29:48.819 INFO:teuthology.orchestra.run.smithi134.stdout:Get:1 https://2.chacra.ceph.com/r/ceph/wip-bharath11-testing-2024-09-18-1115-quincy/7061ea6ca50908d104e9f3979dc615e118d9e153/ubuntu/focal/flavors/default focal/main amd64 librbd1 amd64 17.2.7-1645-g7061ea6c-1focal [3,507 kB] 2024-09-18T10:29:48.862 INFO:teuthology.orchestra.run.smithi134.stdout:Get:2 http://archive.ubuntu.com/ubuntu focal/main amd64 liblttng-ust0 amd64 2.11.0-1 [160 kB] 2024-09-18T10:29:48.994 INFO:teuthology.orchestra.run.smithi134.stdout:Get:3 https://2.chacra.ceph.com/r/ceph/wip-bharath11-testing-2024-09-18-1115-quincy/7061ea6ca50908d104e9f3979dc615e118d9e153/ubuntu/focal/flavors/default focal/main amd64 librados2 amd64 17.2.7-1645-g7061ea6c-1focal [3,562 kB] 2024-09-18T10:29:49.044 INFO:teuthology.orchestra.run.smithi134.stdout:Get:4 https://2.chacra.ceph.com/r/ceph/wip-bharath11-testing-2024-09-18-1115-quincy/7061ea6ca50908d104e9f3979dc615e118d9e153/ubuntu/focal/flavors/default focal/main amd64 libcephfs2 amd64 17.2.7-1645-g7061ea6c-1focal [761 kB] 2024-09-18T10:29:49.056 INFO:teuthology.orchestra.run.smithi134.stdout:Get:5 https://2.chacra.ceph.com/r/ceph/wip-bharath11-testing-2024-09-18-1115-quincy/7061ea6ca50908d104e9f3979dc615e118d9e153/ubuntu/focal/flavors/default focal/main amd64 python3-rados amd64 17.2.7-1645-g7061ea6c-1focal [414 kB] 2024-09-18T10:29:49.098 INFO:teuthology.orchestra.run.smithi134.stdout:Get:6 https://2.chacra.ceph.com/r/ceph/wip-bharath11-testing-2024-09-18-1115-quincy/7061ea6ca50908d104e9f3979dc615e118d9e153/ubuntu/focal/flavors/default focal/main amd64 python3-ceph-argparse all 17.2.7-1645-g7061ea6c-1focal [22.6 kB] 2024-09-18T10:29:49.099 INFO:teuthology.orchestra.run.smithi134.stdout:Get:7 https://2.chacra.ceph.com/r/ceph/wip-bharath11-testing-2024-09-18-1115-quincy/7061ea6ca50908d104e9f3979dc615e118d9e153/ubuntu/focal/flavors/default focal/main amd64 python3-cephfs amd64 17.2.7-1645-g7061ea6c-1focal [200 kB] 2024-09-18T10:29:49.103 INFO:teuthology.orchestra.run.smithi134.stdout:Get:8 https://2.chacra.ceph.com/r/ceph/wip-bharath11-testing-2024-09-18-1115-quincy/7061ea6ca50908d104e9f3979dc615e118d9e153/ubuntu/focal/flavors/default focal/main amd64 python3-ceph-common all 17.2.7-1645-g7061ea6c-1focal [49.1 kB] 2024-09-18T10:29:49.104 INFO:teuthology.orchestra.run.smithi134.stdout:Get:9 https://2.chacra.ceph.com/r/ceph/wip-bharath11-testing-2024-09-18-1115-quincy/7061ea6ca50908d104e9f3979dc615e118d9e153/ubuntu/focal/flavors/default focal/main amd64 python3-rbd amd64 17.2.7-1645-g7061ea6c-1focal [390 kB] 2024-09-18T10:29:49.109 INFO:teuthology.orchestra.run.smithi134.stdout:Get:10 https://2.chacra.ceph.com/r/ceph/wip-bharath11-testing-2024-09-18-1115-quincy/7061ea6ca50908d104e9f3979dc615e118d9e153/ubuntu/focal/flavors/default focal/main amd64 librgw2 amd64 17.2.7-1645-g7061ea6c-1focal [5,637 kB] 2024-09-18T10:29:49.167 INFO:teuthology.orchestra.run.smithi134.stdout:Get:11 https://2.chacra.ceph.com/r/ceph/wip-bharath11-testing-2024-09-18-1115-quincy/7061ea6ca50908d104e9f3979dc615e118d9e153/ubuntu/focal/flavors/default focal/main amd64 python3-rgw amd64 17.2.7-1645-g7061ea6c-1focal [116 kB] 2024-09-18T10:29:49.168 INFO:teuthology.orchestra.run.smithi134.stdout:Get:12 https://2.chacra.ceph.com/r/ceph/wip-bharath11-testing-2024-09-18-1115-quincy/7061ea6ca50908d104e9f3979dc615e118d9e153/ubuntu/focal/flavors/default focal/main amd64 libradosstriper1 amd64 17.2.7-1645-g7061ea6c-1focal [479 kB] 2024-09-18T10:29:49.176 INFO:teuthology.orchestra.run.smithi134.stdout:Get:13 https://2.chacra.ceph.com/r/ceph/wip-bharath11-testing-2024-09-18-1115-quincy/7061ea6ca50908d104e9f3979dc615e118d9e153/ubuntu/focal/flavors/default focal/main amd64 ceph-common amd64 17.2.7-1645-g7061ea6c-1focal [27.0 MB] 2024-09-18T10:29:49.264 INFO:teuthology.orchestra.run.smithi134.stdout:Get:14 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-prettytable all 0.7.2-5 [20.1 kB] 2024-09-18T10:29:49.272 INFO:teuthology.orchestra.run.smithi134.stdout:Get:15 http://archive.ubuntu.com/ubuntu focal/main amd64 liblua5.3-0 amd64 5.3.3-1.1ubuntu2 [116 kB] 2024-09-18T10:29:49.316 INFO:teuthology.orchestra.run.smithi134.stdout:Get:16 http://archive.ubuntu.com/ubuntu focal/main amd64 librabbitmq4 amd64 0.10.0-1 [35.8 kB] 2024-09-18T10:29:49.329 INFO:teuthology.orchestra.run.smithi134.stdout:Get:17 http://archive.ubuntu.com/ubuntu focal/universe amd64 librdkafka1 amd64 1.2.1-1ubuntu1 [410 kB] 2024-09-18T10:29:49.454 INFO:teuthology.orchestra.run.smithi134.stdout:Get:18 http://archive.ubuntu.com/ubuntu focal/universe amd64 libdouble-conversion3 amd64 3.1.5-4ubuntu1 [37.9 kB] 2024-09-18T10:29:49.460 INFO:teuthology.orchestra.run.smithi134.stdout:Get:19 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libpcre2-16-0 amd64 10.34-7ubuntu0.1 [181 kB] 2024-09-18T10:29:49.489 INFO:teuthology.orchestra.run.smithi134.stdout:Get:20 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libqt5core5a amd64 5.12.8+dfsg-0ubuntu2.1 [2,006 kB] 2024-09-18T10:29:49.670 INFO:teuthology.orchestra.run.smithi134.stdout:Get:21 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libqt5dbus5 amd64 5.12.8+dfsg-0ubuntu2.1 [208 kB] 2024-09-18T10:29:49.704 INFO:teuthology.orchestra.run.smithi134.stdout:Get:22 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libqt5network5 amd64 5.12.8+dfsg-0ubuntu2.1 [673 kB] 2024-09-18T10:29:49.745 INFO:teuthology.orchestra.run.smithi134.stdout:Get:23 http://archive.ubuntu.com/ubuntu focal/universe amd64 libthrift-0.13.0 amd64 0.13.0-2build2 [249 kB] 2024-09-18T10:29:49.745 INFO:teuthology.orchestra.run.smithi134.stdout:Get:24 https://2.chacra.ceph.com/r/ceph/wip-bharath11-testing-2024-09-18-1115-quincy/7061ea6ca50908d104e9f3979dc615e118d9e153/ubuntu/focal/flavors/default focal/main amd64 ceph-base amd64 17.2.7-1645-g7061ea6c-1focal [5,177 kB] 2024-09-18T10:29:49.797 INFO:teuthology.orchestra.run.smithi134.stdout:Get:25 http://archive.ubuntu.com/ubuntu focal/main amd64 liboath0 amd64 2.6.1-1.3 [44.7 kB] 2024-09-18T10:29:49.797 INFO:teuthology.orchestra.run.smithi134.stdout:Get:26 https://2.chacra.ceph.com/r/ceph/wip-bharath11-testing-2024-09-18-1115-quincy/7061ea6ca50908d104e9f3979dc615e118d9e153/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-modules-core all 17.2.7-1645-g7061ea6c-1focal [223 kB] 2024-09-18T10:29:49.798 INFO:teuthology.orchestra.run.smithi134.stdout:Get:27 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-cryptography amd64 2.8-3ubuntu0.3 [211 kB] 2024-09-18T10:29:49.799 INFO:teuthology.orchestra.run.smithi134.stdout:Get:28 https://2.chacra.ceph.com/r/ceph/wip-bharath11-testing-2024-09-18-1115-quincy/7061ea6ca50908d104e9f3979dc615e118d9e153/ubuntu/focal/flavors/default focal/main amd64 libsqlite3-mod-ceph amd64 17.2.7-1645-g7061ea6c-1focal [157 kB] 2024-09-18T10:29:49.800 INFO:teuthology.orchestra.run.smithi134.stdout:Get:29 https://2.chacra.ceph.com/r/ceph/wip-bharath11-testing-2024-09-18-1115-quincy/7061ea6ca50908d104e9f3979dc615e118d9e153/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr amd64 17.2.7-1645-g7061ea6c-1focal [1,594 kB] 2024-09-18T10:29:49.803 INFO:teuthology.orchestra.run.smithi134.stdout:Get:30 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-openssl all 19.0.0-1build1 [43.3 kB] 2024-09-18T10:29:49.804 INFO:teuthology.orchestra.run.smithi134.stdout:Get:31 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-packaging all 20.3-1 [26.8 kB] 2024-09-18T10:29:49.806 INFO:teuthology.orchestra.run.smithi134.stdout:Get:32 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-bcrypt amd64 3.1.7-2ubuntu1 [30.4 kB] 2024-09-18T10:29:49.806 INFO:teuthology.orchestra.run.smithi134.stdout:Get:33 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-cherrypy3 all 8.9.1-6 [160 kB] 2024-09-18T10:29:49.810 INFO:teuthology.orchestra.run.smithi134.stdout:Get:34 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-natsort all 7.0.1-1 [29.2 kB] 2024-09-18T10:29:49.819 INFO:teuthology.orchestra.run.smithi134.stdout:Get:35 https://2.chacra.ceph.com/r/ceph/wip-bharath11-testing-2024-09-18-1115-quincy/7061ea6ca50908d104e9f3979dc615e118d9e153/ubuntu/focal/flavors/default focal/main amd64 ceph-mon amd64 17.2.7-1645-g7061ea6c-1focal [4,688 kB] 2024-09-18T10:29:49.833 INFO:teuthology.orchestra.run.smithi134.stdout:Get:36 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-jwt all 1.7.1-2ubuntu2.1 [18.0 kB] 2024-09-18T10:29:49.834 INFO:teuthology.orchestra.run.smithi134.stdout:Get:37 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-logutils all 0.3.3-7 [17.3 kB] 2024-09-18T10:29:49.894 INFO:teuthology.orchestra.run.smithi134.stdout:Get:38 https://2.chacra.ceph.com/r/ceph/wip-bharath11-testing-2024-09-18-1115-quincy/7061ea6ca50908d104e9f3979dc615e118d9e153/ubuntu/focal/flavors/default focal/main amd64 ceph-osd amd64 17.2.7-1645-g7061ea6c-1focal [23.1 MB] 2024-09-18T10:29:49.914 INFO:teuthology.orchestra.run.smithi134.stdout:Get:39 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-markupsafe amd64 1.1.0-1build2 [13.9 kB] 2024-09-18T10:29:49.915 INFO:teuthology.orchestra.run.smithi134.stdout:Get:40 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-mako all 1.1.0+ds1-1ubuntu2.1 [59.2 kB] 2024-09-18T10:29:49.917 INFO:teuthology.orchestra.run.smithi134.stdout:Get:41 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2024-09-18T10:29:49.918 INFO:teuthology.orchestra.run.smithi134.stdout:Get:42 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-singledispatch all 3.4.0.3-2build1 [7,128 B] 2024-09-18T10:29:49.919 INFO:teuthology.orchestra.run.smithi134.stdout:Get:43 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-webob all 1:1.8.5-2ubuntu0.1 [84.5 kB] 2024-09-18T10:29:49.921 INFO:teuthology.orchestra.run.smithi134.stdout:Get:44 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-soupsieve all 1.9.5+dfsg-1 [29.1 kB] 2024-09-18T10:29:49.922 INFO:teuthology.orchestra.run.smithi134.stdout:Get:45 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-bs4 all 4.8.2-1 [83.0 kB] 2024-09-18T10:29:49.924 INFO:teuthology.orchestra.run.smithi134.stdout:Get:46 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-waitress all 1.4.1-1ubuntu0.1 [81.4 kB] 2024-09-18T10:29:49.996 INFO:teuthology.orchestra.run.smithi134.stdout:Get:47 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-tempita all 0.5.2-6 [14.8 kB] 2024-09-18T10:29:49.997 INFO:teuthology.orchestra.run.smithi134.stdout:Get:48 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-paste all 3.4.0+dfsg1-1 [456 kB] 2024-09-18T10:29:50.077 INFO:teuthology.orchestra.run.smithi134.stdout:Get:49 http://archive.ubuntu.com/ubuntu focal/main amd64 python-pastedeploy-tpl all 2.1.0-1 [5,016 B] 2024-09-18T10:29:50.077 INFO:teuthology.orchestra.run.smithi134.stdout:Get:50 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pastedeploy all 2.1.0-1 [26.5 kB] 2024-09-18T10:29:50.078 INFO:teuthology.orchestra.run.smithi134.stdout:Get:51 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-webtest all 2.0.32-2 [28.2 kB] 2024-09-18T10:29:50.079 INFO:teuthology.orchestra.run.smithi134.stdout:Get:52 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pecan all 1.3.3-3ubuntu1 [85.8 kB] 2024-09-18T10:29:50.081 INFO:teuthology.orchestra.run.smithi134.stdout:Get:53 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-werkzeug all 0.16.1+dfsg1-2ubuntu0.2 [183 kB] 2024-09-18T10:29:50.086 INFO:teuthology.orchestra.run.smithi134.stdout:Get:54 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-jinja2 all 2.10.1-2ubuntu0.3 [96.0 kB] 2024-09-18T10:29:50.089 INFO:teuthology.orchestra.run.smithi134.stdout:Get:55 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2024-09-18T10:29:50.090 INFO:teuthology.orchestra.run.smithi134.stdout:Get:56 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 libjs-underscore all 1.9.1~dfsg-1ubuntu0.20.04.1 [99.5 kB] 2024-09-18T10:29:50.158 INFO:teuthology.orchestra.run.smithi134.stdout:Get:57 http://archive.ubuntu.com/ubuntu focal/main amd64 libjs-sphinxdoc all 1.8.5-7ubuntu3 [97.1 kB] 2024-09-18T10:29:50.160 INFO:teuthology.orchestra.run.smithi134.stdout:Get:58 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-routes all 2.4.1-2 [88.1 kB] 2024-09-18T10:29:50.173 INFO:teuthology.orchestra.run.smithi134.stdout:Get:59 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-decorator all 4.4.2-0ubuntu1 [10.3 kB] 2024-09-18T10:29:50.174 INFO:teuthology.orchestra.run.smithi134.stdout:Get:60 http://archive.ubuntu.com/ubuntu focal/universe amd64 liblbfgsb0 amd64 3.0+dfsg.3-7build1 [29.5 kB] 2024-09-18T10:29:50.175 INFO:teuthology.orchestra.run.smithi134.stdout:Get:61 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-scipy amd64 1.3.3-3build1 [11.1 MB] 2024-09-18T10:29:50.356 INFO:teuthology.orchestra.run.smithi134.stdout:Get:62 https://2.chacra.ceph.com/r/ceph/wip-bharath11-testing-2024-09-18-1115-quincy/7061ea6ca50908d104e9f3979dc615e118d9e153/ubuntu/focal/flavors/default focal/main amd64 ceph amd64 17.2.7-1645-g7061ea6c-1focal [3,936 B] 2024-09-18T10:29:50.357 INFO:teuthology.orchestra.run.smithi134.stdout:Get:63 https://2.chacra.ceph.com/r/ceph/wip-bharath11-testing-2024-09-18-1115-quincy/7061ea6ca50908d104e9f3979dc615e118d9e153/ubuntu/focal/flavors/default focal/main amd64 ceph-fuse amd64 17.2.7-1645-g7061ea6c-1focal [872 kB] 2024-09-18T10:29:50.370 INFO:teuthology.orchestra.run.smithi134.stdout:Get:64 https://2.chacra.ceph.com/r/ceph/wip-bharath11-testing-2024-09-18-1115-quincy/7061ea6ca50908d104e9f3979dc615e118d9e153/ubuntu/focal/flavors/default focal/main amd64 ceph-mds amd64 17.2.7-1645-g7061ea6c-1focal [2,416 kB] 2024-09-18T10:29:50.388 INFO:teuthology.orchestra.run.smithi134.stdout:Get:65 https://2.chacra.ceph.com/r/ceph/wip-bharath11-testing-2024-09-18-1115-quincy/7061ea6ca50908d104e9f3979dc615e118d9e153/ubuntu/focal/flavors/default focal/main amd64 cephadm amd64 17.2.7-1645-g7061ea6c-1focal [81.4 kB] 2024-09-18T10:29:50.389 INFO:teuthology.orchestra.run.smithi134.stdout:Get:66 https://2.chacra.ceph.com/r/ceph/wip-bharath11-testing-2024-09-18-1115-quincy/7061ea6ca50908d104e9f3979dc615e118d9e153/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-cephadm all 17.2.7-1645-g7061ea6c-1focal [120 kB] 2024-09-18T10:29:50.390 INFO:teuthology.orchestra.run.smithi134.stdout:Get:67 https://2.chacra.ceph.com/r/ceph/wip-bharath11-testing-2024-09-18-1115-quincy/7061ea6ca50908d104e9f3979dc615e118d9e153/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-dashboard all 17.2.7-1645-g7061ea6c-1focal [6,653 kB] 2024-09-18T10:29:50.456 INFO:teuthology.orchestra.run.smithi134.stdout:Get:68 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-sklearn-lib amd64 0.22.2.post1+dfsg-5 [2,358 kB] 2024-09-18T10:29:50.564 INFO:teuthology.orchestra.run.smithi134.stdout:Get:69 https://2.chacra.ceph.com/r/ceph/wip-bharath11-testing-2024-09-18-1115-quincy/7061ea6ca50908d104e9f3979dc615e118d9e153/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-diskprediction-local all 17.2.7-1645-g7061ea6c-1focal [7,884 kB] 2024-09-18T10:29:50.564 INFO:teuthology.orchestra.run.smithi134.stdout:Get:70 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-joblib all 0.14.0-3 [192 kB] 2024-09-18T10:29:50.565 INFO:teuthology.orchestra.run.smithi134.stdout:Get:71 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-sklearn all 0.22.2.post1+dfsg-5 [1,773 kB] 2024-09-18T10:29:50.603 INFO:teuthology.orchestra.run.smithi134.stdout:Get:72 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-cachetools all 4.0.0-1 [9,820 B] 2024-09-18T10:29:50.603 INFO:teuthology.orchestra.run.smithi134.stdout:Get:73 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyasn1 all 0.4.2-3build1 [46.7 kB] 2024-09-18T10:29:50.603 INFO:teuthology.orchestra.run.smithi134.stdout:Get:74 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyasn1-modules all 0.2.1-0.2build1 [32.9 kB] 2024-09-18T10:29:50.604 INFO:teuthology.orchestra.run.smithi134.stdout:Get:75 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-rsa all 4.0-3 [24.7 kB] 2024-09-18T10:29:50.604 INFO:teuthology.orchestra.run.smithi134.stdout:Get:76 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-google-auth all 1.5.1-2 [35.6 kB] 2024-09-18T10:29:50.604 INFO:teuthology.orchestra.run.smithi134.stdout:Get:77 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-blinker all 1.4+dfsg1-0.3ubuntu1 [13.2 kB] 2024-09-18T10:29:50.605 INFO:teuthology.orchestra.run.smithi134.stdout:Get:78 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-oauthlib all 3.1.0-1ubuntu2 [84.8 kB] 2024-09-18T10:29:50.645 INFO:teuthology.orchestra.run.smithi134.stdout:Get:79 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-requests-oauthlib all 1.0.0-1.1build1 [16.8 kB] 2024-09-18T10:29:50.657 INFO:teuthology.orchestra.run.smithi134.stdout:Get:80 https://2.chacra.ceph.com/r/ceph/wip-bharath11-testing-2024-09-18-1115-quincy/7061ea6ca50908d104e9f3979dc615e118d9e153/ubuntu/focal/flavors/default focal/main amd64 ceph-mgr-k8sevents all 17.2.7-1645-g7061ea6c-1focal [4,056 B] 2024-09-18T10:29:50.658 INFO:teuthology.orchestra.run.smithi134.stdout:Get:81 https://2.chacra.ceph.com/r/ceph/wip-bharath11-testing-2024-09-18-1115-quincy/7061ea6ca50908d104e9f3979dc615e118d9e153/ubuntu/focal/flavors/default focal/main amd64 ceph-test amd64 17.2.7-1645-g7061ea6c-1focal [53.4 MB] 2024-09-18T10:29:50.687 INFO:teuthology.orchestra.run.smithi134.stdout:Get:82 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-websocket all 0.53.0-2ubuntu1 [32.3 kB] 2024-09-18T10:29:50.688 INFO:teuthology.orchestra.run.smithi134.stdout:Get:83 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-kubernetes all 7.0.0~a1-2 [271 kB] 2024-09-18T10:29:50.694 INFO:teuthology.orchestra.run.smithi134.stdout:Get:84 http://archive.ubuntu.com/ubuntu focal/universe amd64 libonig5 amd64 6.9.4-1 [142 kB] 2024-09-18T10:29:50.697 INFO:teuthology.orchestra.run.smithi134.stdout:Get:85 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 libjq1 amd64 1.6-1ubuntu0.20.04.1 [121 kB] 2024-09-18T10:29:50.700 INFO:teuthology.orchestra.run.smithi134.stdout:Get:86 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 jq amd64 1.6-1ubuntu0.20.04.1 [50.2 kB] 2024-09-18T10:29:50.702 INFO:teuthology.orchestra.run.smithi134.stdout:Get:87 http://archive.ubuntu.com/ubuntu focal/main amd64 socat amd64 1.7.3.3-2 [323 kB] 2024-09-18T10:29:50.731 INFO:teuthology.orchestra.run.smithi134.stdout:Get:88 http://archive.ubuntu.com/ubuntu focal/universe amd64 xmlstarlet amd64 1.6.1-2build1 [267 kB] 2024-09-18T10:29:50.736 INFO:teuthology.orchestra.run.smithi134.stdout:Get:89 http://archive.ubuntu.com/ubuntu focal/universe amd64 fonts-mathjax all 2.7.4+dfsg-1 [2,208 kB] 2024-09-18T10:29:50.777 INFO:teuthology.orchestra.run.smithi134.stdout:Get:90 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 nvme-cli amd64 1.9-1ubuntu0.2 [283 kB] 2024-09-18T10:29:50.812 INFO:teuthology.orchestra.run.smithi134.stdout:Get:91 http://archive.ubuntu.com/ubuntu focal/universe amd64 libjs-mathjax all 2.7.4+dfsg-1 [5,654 kB] 2024-09-18T10:29:50.975 INFO:teuthology.orchestra.run.smithi134.stdout:Get:92 http://archive.ubuntu.com/ubuntu focal/universe amd64 python-natsort-doc all 7.0.1-1 [103 kB] 2024-09-18T10:29:50.976 INFO:teuthology.orchestra.run.smithi134.stdout:Get:93 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-atomicwrites all 1.1.5-2build1 [7,016 B] 2024-09-18T10:29:50.976 INFO:teuthology.orchestra.run.smithi134.stdout:Get:94 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-attr all 19.3.0-2 [33.9 kB] 2024-09-18T10:29:50.977 INFO:teuthology.orchestra.run.smithi134.stdout:Get:95 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-webencodings all 0.5.1-1ubuntu1 [11.0 kB] 2024-09-18T10:29:50.977 INFO:teuthology.orchestra.run.smithi134.stdout:Get:96 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-html5lib all 1.0.1-2 [84.3 kB] 2024-09-18T10:29:50.979 INFO:teuthology.orchestra.run.smithi134.stdout:Get:97 http://archive.ubuntu.com/ubuntu focal-updates/main amd64 python3-lxml amd64 4.5.0-1ubuntu0.5 [1,384 kB] 2024-09-18T10:29:50.995 INFO:teuthology.orchestra.run.smithi134.stdout:Get:98 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2024-09-18T10:29:50.995 INFO:teuthology.orchestra.run.smithi134.stdout:Get:99 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-pluggy all 0.13.0-2 [18.4 kB] 2024-09-18T10:29:51.040 INFO:teuthology.orchestra.run.smithi134.stdout:Get:100 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-psutil amd64 5.5.1-1ubuntu4 [141 kB] 2024-09-18T10:29:51.057 INFO:teuthology.orchestra.run.smithi134.stdout:Get:101 http://archive.ubuntu.com/ubuntu focal-updates/universe amd64 python3-py all 1.8.1-1ubuntu0.1 [65.6 kB] 2024-09-18T10:29:51.138 INFO:teuthology.orchestra.run.smithi134.stdout:Get:102 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-pyinotify all 0.9.6-1.2ubuntu1 [24.8 kB] 2024-09-18T10:29:51.139 INFO:teuthology.orchestra.run.smithi134.stdout:Get:103 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-wcwidth all 0.1.8+dfsg1-3 [17.4 kB] 2024-09-18T10:29:51.139 INFO:teuthology.orchestra.run.smithi134.stdout:Get:104 http://archive.ubuntu.com/ubuntu focal/universe amd64 python3-pytest all 4.6.9-1 [178 kB] 2024-09-18T10:29:51.143 INFO:teuthology.orchestra.run.smithi134.stdout:Get:105 http://archive.ubuntu.com/ubuntu focal/main amd64 python3-simplejson amd64 3.16.0-2ubuntu2 [50.8 kB] 2024-09-18T10:29:51.144 INFO:teuthology.orchestra.run.smithi134.stdout:Get:106 http://archive.ubuntu.com/ubuntu focal/universe amd64 qttranslations5-l10n all 5.12.8-0ubuntu1 [1,486 kB] 2024-09-18T10:29:51.809 INFO:teuthology.orchestra.run.smithi134.stdout:Get:107 https://2.chacra.ceph.com/r/ceph/wip-bharath11-testing-2024-09-18-1115-quincy/7061ea6ca50908d104e9f3979dc615e118d9e153/ubuntu/focal/flavors/default focal/main amd64 ceph-volume all 17.2.7-1645-g7061ea6c-1focal [231 kB] 2024-09-18T10:29:51.813 INFO:teuthology.orchestra.run.smithi134.stdout:Get:108 https://2.chacra.ceph.com/r/ceph/wip-bharath11-testing-2024-09-18-1115-quincy/7061ea6ca50908d104e9f3979dc615e118d9e153/ubuntu/focal/flavors/default focal/main amd64 libcephfs-dev amd64 17.2.7-1645-g7061ea6c-1focal [21.9 kB] 2024-09-18T10:29:51.814 INFO:teuthology.orchestra.run.smithi134.stdout:Get:109 https://2.chacra.ceph.com/r/ceph/wip-bharath11-testing-2024-09-18-1115-quincy/7061ea6ca50908d104e9f3979dc615e118d9e153/ubuntu/focal/flavors/default focal/main amd64 radosgw amd64 17.2.7-1645-g7061ea6c-1focal [17.2 MB] 2024-09-18T10:29:52.079 INFO:teuthology.orchestra.run.smithi134.stdout:Get:110 https://2.chacra.ceph.com/r/ceph/wip-bharath11-testing-2024-09-18-1115-quincy/7061ea6ca50908d104e9f3979dc615e118d9e153/ubuntu/focal/flavors/default focal/main amd64 rbd-fuse amd64 17.2.7-1645-g7061ea6c-1focal [80.6 kB] 2024-09-18T10:29:52.726 INFO:teuthology.orchestra.run.smithi134.stdout:Fetched 201 MB in 3s (59.1 MB/s) 2024-09-18T10:29:52.944 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package liblttng-ust0:amd64. 2024-09-18T10:29:52.999 INFO:teuthology.orchestra.run.smithi134.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 141720 files and directories currently installed.) 2024-09-18T10:29:53.003 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../000-liblttng-ust0_2.11.0-1_amd64.deb ... 2024-09-18T10:29:53.084 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking liblttng-ust0:amd64 (2.11.0-1) ... 2024-09-18T10:29:53.456 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../001-librbd1_17.2.7-1645-g7061ea6c-1focal_amd64.deb ... 2024-09-18T10:29:53.587 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking librbd1 (17.2.7-1645-g7061ea6c-1focal) over (15.2.17-0ubuntu0.20.04.6) ... 2024-09-18T10:29:54.282 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../002-librados2_17.2.7-1645-g7061ea6c-1focal_amd64.deb ... 2024-09-18T10:29:54.441 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking librados2 (17.2.7-1645-g7061ea6c-1focal) over (15.2.17-0ubuntu0.20.04.6) ... 2024-09-18T10:29:55.145 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package libcephfs2. 2024-09-18T10:29:55.165 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../003-libcephfs2_17.2.7-1645-g7061ea6c-1focal_amd64.deb ... 2024-09-18T10:29:55.203 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking libcephfs2 (17.2.7-1645-g7061ea6c-1focal) ... 2024-09-18T10:29:55.497 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package python3-rados. 2024-09-18T10:29:55.514 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../004-python3-rados_17.2.7-1645-g7061ea6c-1focal_amd64.deb ... 2024-09-18T10:29:55.572 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking python3-rados (17.2.7-1645-g7061ea6c-1focal) ... 2024-09-18T10:29:55.941 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package python3-ceph-argparse. 2024-09-18T10:29:55.958 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../005-python3-ceph-argparse_17.2.7-1645-g7061ea6c-1focal_all.deb ... 2024-09-18T10:29:56.000 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking python3-ceph-argparse (17.2.7-1645-g7061ea6c-1focal) ... 2024-09-18T10:29:56.294 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package python3-cephfs. 2024-09-18T10:29:56.310 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../006-python3-cephfs_17.2.7-1645-g7061ea6c-1focal_amd64.deb ... 2024-09-18T10:29:56.352 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking python3-cephfs (17.2.7-1645-g7061ea6c-1focal) ... 2024-09-18T10:29:56.705 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package python3-ceph-common. 2024-09-18T10:29:56.724 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../007-python3-ceph-common_17.2.7-1645-g7061ea6c-1focal_all.deb ... 2024-09-18T10:29:56.787 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking python3-ceph-common (17.2.7-1645-g7061ea6c-1focal) ... 2024-09-18T10:29:57.124 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package python3-prettytable. 2024-09-18T10:29:57.140 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../008-python3-prettytable_0.7.2-5_all.deb ... 2024-09-18T10:29:57.174 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking python3-prettytable (0.7.2-5) ... 2024-09-18T10:29:57.410 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package python3-rbd. 2024-09-18T10:29:57.426 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../009-python3-rbd_17.2.7-1645-g7061ea6c-1focal_amd64.deb ... 2024-09-18T10:29:57.459 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking python3-rbd (17.2.7-1645-g7061ea6c-1focal) ... 2024-09-18T10:29:57.829 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package liblua5.3-0:amd64. 2024-09-18T10:29:57.848 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../010-liblua5.3-0_5.3.3-1.1ubuntu2_amd64.deb ... 2024-09-18T10:29:57.887 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking liblua5.3-0:amd64 (5.3.3-1.1ubuntu2) ... 2024-09-18T10:29:58.290 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package librabbitmq4:amd64. 2024-09-18T10:29:58.308 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../011-librabbitmq4_0.10.0-1_amd64.deb ... 2024-09-18T10:29:58.348 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking librabbitmq4:amd64 (0.10.0-1) ... 2024-09-18T10:29:58.667 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package librdkafka1:amd64. 2024-09-18T10:29:58.683 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../012-librdkafka1_1.2.1-1ubuntu1_amd64.deb ... 2024-09-18T10:29:58.717 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking librdkafka1:amd64 (1.2.1-1ubuntu1) ... 2024-09-18T10:29:59.112 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2024-09-18T10:29:59.129 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../013-libdouble-conversion3_3.1.5-4ubuntu1_amd64.deb ... 2024-09-18T10:29:59.187 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking libdouble-conversion3:amd64 (3.1.5-4ubuntu1) ... 2024-09-18T10:29:59.548 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2024-09-18T10:29:59.565 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../014-libpcre2-16-0_10.34-7ubuntu0.1_amd64.deb ... 2024-09-18T10:29:59.606 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking libpcre2-16-0:amd64 (10.34-7ubuntu0.1) ... 2024-09-18T10:30:00.135 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package libqt5core5a:amd64. 2024-09-18T10:30:00.150 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../015-libqt5core5a_5.12.8+dfsg-0ubuntu2.1_amd64.deb ... 2024-09-18T10:30:00.201 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking libqt5core5a:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2024-09-18T10:30:00.729 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2024-09-18T10:30:00.746 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../016-libqt5dbus5_5.12.8+dfsg-0ubuntu2.1_amd64.deb ... 2024-09-18T10:30:00.787 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking libqt5dbus5:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2024-09-18T10:30:01.132 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package libqt5network5:amd64. 2024-09-18T10:30:01.149 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../017-libqt5network5_5.12.8+dfsg-0ubuntu2.1_amd64.deb ... 2024-09-18T10:30:01.224 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking libqt5network5:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2024-09-18T10:30:01.651 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package libthrift-0.13.0. 2024-09-18T10:30:01.671 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../018-libthrift-0.13.0_0.13.0-2build2_amd64.deb ... 2024-09-18T10:30:01.710 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking libthrift-0.13.0 (0.13.0-2build2) ... 2024-09-18T10:30:02.115 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package librgw2. 2024-09-18T10:30:02.135 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../019-librgw2_17.2.7-1645-g7061ea6c-1focal_amd64.deb ... 2024-09-18T10:30:02.179 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking librgw2 (17.2.7-1645-g7061ea6c-1focal) ... 2024-09-18T10:30:02.957 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package python3-rgw. 2024-09-18T10:30:02.976 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../020-python3-rgw_17.2.7-1645-g7061ea6c-1focal_amd64.deb ... 2024-09-18T10:30:03.006 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking python3-rgw (17.2.7-1645-g7061ea6c-1focal) ... 2024-09-18T10:30:03.392 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package liboath0. 2024-09-18T10:30:03.409 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../021-liboath0_2.6.1-1.3_amd64.deb ... 2024-09-18T10:30:03.450 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking liboath0 (2.6.1-1.3) ... 2024-09-18T10:30:03.795 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package libradosstriper1. 2024-09-18T10:30:03.813 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../022-libradosstriper1_17.2.7-1645-g7061ea6c-1focal_amd64.deb ... 2024-09-18T10:30:03.862 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking libradosstriper1 (17.2.7-1645-g7061ea6c-1focal) ... 2024-09-18T10:30:04.348 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package ceph-common. 2024-09-18T10:30:04.363 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../023-ceph-common_17.2.7-1645-g7061ea6c-1focal_amd64.deb ... 2024-09-18T10:30:04.398 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking ceph-common (17.2.7-1645-g7061ea6c-1focal) ... 2024-09-18T10:30:07.153 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package ceph-base. 2024-09-18T10:30:07.169 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../024-ceph-base_17.2.7-1645-g7061ea6c-1focal_amd64.deb ... 2024-09-18T10:30:07.221 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking ceph-base (17.2.7-1645-g7061ea6c-1focal) ... 2024-09-18T10:30:08.132 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package python3-cryptography. 2024-09-18T10:30:08.152 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../025-python3-cryptography_2.8-3ubuntu0.3_amd64.deb ... 2024-09-18T10:30:08.215 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking python3-cryptography (2.8-3ubuntu0.3) ... 2024-09-18T10:30:08.627 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package python3-openssl. 2024-09-18T10:30:08.643 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../026-python3-openssl_19.0.0-1build1_all.deb ... 2024-09-18T10:30:08.685 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking python3-openssl (19.0.0-1build1) ... 2024-09-18T10:30:09.046 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package python3-packaging. 2024-09-18T10:30:09.062 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../027-python3-packaging_20.3-1_all.deb ... 2024-09-18T10:30:09.130 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking python3-packaging (20.3-1) ... 2024-09-18T10:30:09.407 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2024-09-18T10:30:09.426 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../028-ceph-mgr-modules-core_17.2.7-1645-g7061ea6c-1focal_all.deb ... 2024-09-18T10:30:09.465 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking ceph-mgr-modules-core (17.2.7-1645-g7061ea6c-1focal) ... 2024-09-18T10:30:09.872 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package python3-bcrypt. 2024-09-18T10:30:09.890 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../029-python3-bcrypt_3.1.7-2ubuntu1_amd64.deb ... 2024-09-18T10:30:09.930 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking python3-bcrypt (3.1.7-2ubuntu1) ... 2024-09-18T10:30:10.300 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package python3-cherrypy3. 2024-09-18T10:30:10.320 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../030-python3-cherrypy3_8.9.1-6_all.deb ... 2024-09-18T10:30:10.358 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking python3-cherrypy3 (8.9.1-6) ... 2024-09-18T10:30:10.711 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package python3-natsort. 2024-09-18T10:30:10.727 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../031-python3-natsort_7.0.1-1_all.deb ... 2024-09-18T10:30:10.769 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking python3-natsort (7.0.1-1) ... 2024-09-18T10:30:11.114 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package python3-jwt. 2024-09-18T10:30:11.130 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../032-python3-jwt_1.7.1-2ubuntu2.1_all.deb ... 2024-09-18T10:30:11.206 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking python3-jwt (1.7.1-2ubuntu2.1) ... 2024-09-18T10:30:11.550 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package python3-logutils. 2024-09-18T10:30:11.566 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../033-python3-logutils_0.3.3-7_all.deb ... 2024-09-18T10:30:11.608 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking python3-logutils (0.3.3-7) ... 2024-09-18T10:30:11.944 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package python3-markupsafe. 2024-09-18T10:30:11.961 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../034-python3-markupsafe_1.1.0-1build2_amd64.deb ... 2024-09-18T10:30:12.003 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking python3-markupsafe (1.1.0-1build2) ... 2024-09-18T10:30:12.339 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package python3-mako. 2024-09-18T10:30:12.354 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../035-python3-mako_1.1.0+ds1-1ubuntu2.1_all.deb ... 2024-09-18T10:30:12.397 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking python3-mako (1.1.0+ds1-1ubuntu2.1) ... 2024-09-18T10:30:12.767 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package python3-simplegeneric. 2024-09-18T10:30:12.783 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../036-python3-simplegeneric_0.8.1-3_all.deb ... 2024-09-18T10:30:12.825 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2024-09-18T10:30:13.144 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package python3-singledispatch. 2024-09-18T10:30:13.160 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../037-python3-singledispatch_3.4.0.3-2build1_all.deb ... 2024-09-18T10:30:13.202 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking python3-singledispatch (3.4.0.3-2build1) ... 2024-09-18T10:30:13.530 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package python3-webob. 2024-09-18T10:30:13.546 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../038-python3-webob_1%3a1.8.5-2ubuntu0.1_all.deb ... 2024-09-18T10:30:13.597 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking python3-webob (1:1.8.5-2ubuntu0.1) ... 2024-09-18T10:30:13.958 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package python3-soupsieve. 2024-09-18T10:30:13.977 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../039-python3-soupsieve_1.9.5+dfsg-1_all.deb ... 2024-09-18T10:30:14.016 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking python3-soupsieve (1.9.5+dfsg-1) ... 2024-09-18T10:30:14.344 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package python3-bs4. 2024-09-18T10:30:14.360 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../040-python3-bs4_4.8.2-1_all.deb ... 2024-09-18T10:30:14.402 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking python3-bs4 (4.8.2-1) ... 2024-09-18T10:30:14.824 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package python3-waitress. 2024-09-18T10:30:14.844 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../041-python3-waitress_1.4.1-1ubuntu0.1_all.deb ... 2024-09-18T10:30:14.895 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking python3-waitress (1.4.1-1ubuntu0.1) ... 2024-09-18T10:30:15.254 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package python3-tempita. 2024-09-18T10:30:15.270 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../042-python3-tempita_0.5.2-6_all.deb ... 2024-09-18T10:30:15.312 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking python3-tempita (0.5.2-6) ... 2024-09-18T10:30:15.717 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package python3-paste. 2024-09-18T10:30:15.736 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../043-python3-paste_3.4.0+dfsg1-1_all.deb ... 2024-09-18T10:30:15.774 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking python3-paste (3.4.0+dfsg1-1) ... 2024-09-18T10:30:16.110 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2024-09-18T10:30:16.129 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../044-python-pastedeploy-tpl_2.1.0-1_all.deb ... 2024-09-18T10:30:16.168 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking python-pastedeploy-tpl (2.1.0-1) ... 2024-09-18T10:30:16.496 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package python3-pastedeploy. 2024-09-18T10:30:16.513 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../045-python3-pastedeploy_2.1.0-1_all.deb ... 2024-09-18T10:30:16.554 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking python3-pastedeploy (2.1.0-1) ... 2024-09-18T10:30:16.899 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package python3-webtest. 2024-09-18T10:30:16.917 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../046-python3-webtest_2.0.32-2_all.deb ... 2024-09-18T10:30:17.007 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking python3-webtest (2.0.32-2) ... 2024-09-18T10:30:17.335 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package python3-pecan. 2024-09-18T10:30:17.353 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../047-python3-pecan_1.3.3-3ubuntu1_all.deb ... 2024-09-18T10:30:17.393 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking python3-pecan (1.3.3-3ubuntu1) ... 2024-09-18T10:30:17.764 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package python3-werkzeug. 2024-09-18T10:30:17.782 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../048-python3-werkzeug_0.16.1+dfsg1-2ubuntu0.2_all.deb ... 2024-09-18T10:30:17.839 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking python3-werkzeug (0.16.1+dfsg1-2ubuntu0.2) ... 2024-09-18T10:30:18.179 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2024-09-18T10:30:18.195 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../049-libsqlite3-mod-ceph_17.2.7-1645-g7061ea6c-1focal_amd64.deb ... 2024-09-18T10:30:18.229 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking libsqlite3-mod-ceph (17.2.7-1645-g7061ea6c-1focal) ... 2024-09-18T10:30:18.607 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package ceph-mgr. 2024-09-18T10:30:18.625 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../050-ceph-mgr_17.2.7-1645-g7061ea6c-1focal_amd64.deb ... 2024-09-18T10:30:18.665 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking ceph-mgr (17.2.7-1645-g7061ea6c-1focal) ... 2024-09-18T10:30:19.175 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package ceph-mon. 2024-09-18T10:30:19.191 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../051-ceph-mon_17.2.7-1645-g7061ea6c-1focal_amd64.deb ... 2024-09-18T10:30:19.233 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking ceph-mon (17.2.7-1645-g7061ea6c-1focal) ... 2024-09-18T10:30:20.070 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package ceph-osd. 2024-09-18T10:30:20.086 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../052-ceph-osd_17.2.7-1645-g7061ea6c-1focal_amd64.deb ... 2024-09-18T10:30:20.128 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking ceph-osd (17.2.7-1645-g7061ea6c-1focal) ... 2024-09-18T10:30:23.190 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package ceph. 2024-09-18T10:30:23.206 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../053-ceph_17.2.7-1645-g7061ea6c-1focal_amd64.deb ... 2024-09-18T10:30:23.267 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking ceph (17.2.7-1645-g7061ea6c-1focal) ... 2024-09-18T10:30:23.618 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package ceph-fuse. 2024-09-18T10:30:23.634 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../054-ceph-fuse_17.2.7-1645-g7061ea6c-1focal_amd64.deb ... 2024-09-18T10:30:23.676 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking ceph-fuse (17.2.7-1645-g7061ea6c-1focal) ... 2024-09-18T10:30:24.129 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package ceph-mds. 2024-09-18T10:30:24.146 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../055-ceph-mds_17.2.7-1645-g7061ea6c-1focal_amd64.deb ... 2024-09-18T10:30:24.212 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking ceph-mds (17.2.7-1645-g7061ea6c-1focal) ... 2024-09-18T10:30:24.815 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package cephadm. 2024-09-18T10:30:24.833 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../056-cephadm_17.2.7-1645-g7061ea6c-1focal_amd64.deb ... 2024-09-18T10:30:24.873 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking cephadm (17.2.7-1645-g7061ea6c-1focal) ... 2024-09-18T10:30:25.226 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package python3-jinja2. 2024-09-18T10:30:25.242 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../057-python3-jinja2_2.10.1-2ubuntu0.3_all.deb ... 2024-09-18T10:30:25.284 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking python3-jinja2 (2.10.1-2ubuntu0.3) ... 2024-09-18T10:30:25.637 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2024-09-18T10:30:25.656 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../058-ceph-mgr-cephadm_17.2.7-1645-g7061ea6c-1focal_all.deb ... 2024-09-18T10:30:25.711 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking ceph-mgr-cephadm (17.2.7-1645-g7061ea6c-1focal) ... 2024-09-18T10:30:26.073 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package python3-repoze.lru. 2024-09-18T10:30:26.090 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../059-python3-repoze.lru_0.7-2_all.deb ... 2024-09-18T10:30:26.132 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2024-09-18T10:30:26.417 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package libjs-underscore. 2024-09-18T10:30:26.433 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../060-libjs-underscore_1.9.1~dfsg-1ubuntu0.20.04.1_all.deb ... 2024-09-18T10:30:26.475 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking libjs-underscore (1.9.1~dfsg-1ubuntu0.20.04.1) ... 2024-09-18T10:30:26.769 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package libjs-sphinxdoc. 2024-09-18T10:30:26.786 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../061-libjs-sphinxdoc_1.8.5-7ubuntu3_all.deb ... 2024-09-18T10:30:26.827 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking libjs-sphinxdoc (1.8.5-7ubuntu3) ... 2024-09-18T10:30:27.171 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package python3-routes. 2024-09-18T10:30:27.187 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../062-python3-routes_2.4.1-2_all.deb ... 2024-09-18T10:30:27.254 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking python3-routes (2.4.1-2) ... 2024-09-18T10:30:27.608 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2024-09-18T10:30:27.628 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../063-ceph-mgr-dashboard_17.2.7-1645-g7061ea6c-1focal_all.deb ... 2024-09-18T10:30:27.666 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking ceph-mgr-dashboard (17.2.7-1645-g7061ea6c-1focal) ... 2024-09-18T10:30:29.029 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package python3-decorator. 2024-09-18T10:30:29.049 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../064-python3-decorator_4.4.2-0ubuntu1_all.deb ... 2024-09-18T10:30:29.087 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking python3-decorator (4.4.2-0ubuntu1) ... 2024-09-18T10:30:29.432 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package liblbfgsb0:amd64. 2024-09-18T10:30:29.448 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../065-liblbfgsb0_3.0+dfsg.3-7build1_amd64.deb ... 2024-09-18T10:30:29.490 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking liblbfgsb0:amd64 (3.0+dfsg.3-7build1) ... 2024-09-18T10:30:29.844 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package python3-scipy. 2024-09-18T10:30:29.863 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../066-python3-scipy_1.3.3-3build1_amd64.deb ... 2024-09-18T10:30:29.935 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking python3-scipy (1.3.3-3build1) ... 2024-09-18T10:30:31.286 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package python3-sklearn-lib. 2024-09-18T10:30:31.305 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../067-python3-sklearn-lib_0.22.2.post1+dfsg-5_amd64.deb ... 2024-09-18T10:30:31.377 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking python3-sklearn-lib (0.22.2.post1+dfsg-5) ... 2024-09-18T10:30:31.977 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package python3-joblib. 2024-09-18T10:30:31.997 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../068-python3-joblib_0.14.0-3_all.deb ... 2024-09-18T10:30:32.046 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking python3-joblib (0.14.0-3) ... 2024-09-18T10:30:32.425 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package python3-sklearn. 2024-09-18T10:30:32.442 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../069-python3-sklearn_0.22.2.post1+dfsg-5_all.deb ... 2024-09-18T10:30:32.483 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking python3-sklearn (0.22.2.post1+dfsg-5) ... 2024-09-18T10:30:33.224 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2024-09-18T10:30:33.240 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../070-ceph-mgr-diskprediction-local_17.2.7-1645-g7061ea6c-1focal_all.deb ... 2024-09-18T10:30:33.274 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking ceph-mgr-diskprediction-local (17.2.7-1645-g7061ea6c-1focal) ... 2024-09-18T10:30:34.452 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package python3-cachetools. 2024-09-18T10:30:34.472 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../071-python3-cachetools_4.0.0-1_all.deb ... 2024-09-18T10:30:34.510 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking python3-cachetools (4.0.0-1) ... 2024-09-18T10:30:34.838 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package python3-pyasn1. 2024-09-18T10:30:34.854 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../072-python3-pyasn1_0.4.2-3build1_all.deb ... 2024-09-18T10:30:34.896 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking python3-pyasn1 (0.4.2-3build1) ... 2024-09-18T10:30:35.241 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package python3-pyasn1-modules. 2024-09-18T10:30:35.260 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../073-python3-pyasn1-modules_0.2.1-0.2build1_all.deb ... 2024-09-18T10:30:35.332 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking python3-pyasn1-modules (0.2.1-0.2build1) ... 2024-09-18T10:30:35.685 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package python3-rsa. 2024-09-18T10:30:35.702 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../074-python3-rsa_4.0-3_all.deb ... 2024-09-18T10:30:35.743 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking python3-rsa (4.0-3) ... 2024-09-18T10:30:36.079 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package python3-google-auth. 2024-09-18T10:30:36.097 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../075-python3-google-auth_1.5.1-2_all.deb ... 2024-09-18T10:30:36.137 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking python3-google-auth (1.5.1-2) ... 2024-09-18T10:30:36.500 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package python3-blinker. 2024-09-18T10:30:36.520 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../076-python3-blinker_1.4+dfsg1-0.3ubuntu1_all.deb ... 2024-09-18T10:30:36.558 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking python3-blinker (1.4+dfsg1-0.3ubuntu1) ... 2024-09-18T10:30:36.885 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package python3-oauthlib. 2024-09-18T10:30:36.902 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../077-python3-oauthlib_3.1.0-1ubuntu2_all.deb ... 2024-09-18T10:30:36.943 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking python3-oauthlib (3.1.0-1ubuntu2) ... 2024-09-18T10:30:37.280 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package python3-requests-oauthlib. 2024-09-18T10:30:37.300 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../078-python3-requests-oauthlib_1.0.0-1.1build1_all.deb ... 2024-09-18T10:30:37.363 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking python3-requests-oauthlib (1.0.0-1.1build1) ... 2024-09-18T10:30:37.725 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package python3-websocket. 2024-09-18T10:30:37.744 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../079-python3-websocket_0.53.0-2ubuntu1_all.deb ... 2024-09-18T10:30:37.783 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking python3-websocket (0.53.0-2ubuntu1) ... 2024-09-18T10:30:38.103 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package python3-kubernetes. 2024-09-18T10:30:38.122 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../080-python3-kubernetes_7.0.0~a1-2_all.deb ... 2024-09-18T10:30:38.189 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking python3-kubernetes (7.0.0~a1-2) ... 2024-09-18T10:30:39.121 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2024-09-18T10:30:39.139 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../081-ceph-mgr-k8sevents_17.2.7-1645-g7061ea6c-1focal_all.deb ... 2024-09-18T10:30:39.179 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking ceph-mgr-k8sevents (17.2.7-1645-g7061ea6c-1focal) ... 2024-09-18T10:30:39.507 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package libonig5:amd64. 2024-09-18T10:30:39.524 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../082-libonig5_6.9.4-1_amd64.deb ... 2024-09-18T10:30:39.565 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking libonig5:amd64 (6.9.4-1) ... 2024-09-18T10:30:39.935 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package libjq1:amd64. 2024-09-18T10:30:39.953 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../083-libjq1_1.6-1ubuntu0.20.04.1_amd64.deb ... 2024-09-18T10:30:39.993 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking libjq1:amd64 (1.6-1ubuntu0.20.04.1) ... 2024-09-18T10:30:40.246 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package jq. 2024-09-18T10:30:40.263 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../084-jq_1.6-1ubuntu0.20.04.1_amd64.deb ... 2024-09-18T10:30:40.304 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking jq (1.6-1ubuntu0.20.04.1) ... 2024-09-18T10:30:40.581 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package socat. 2024-09-18T10:30:40.598 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../085-socat_1.7.3.3-2_amd64.deb ... 2024-09-18T10:30:40.639 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking socat (1.7.3.3-2) ... 2024-09-18T10:30:41.002 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package xmlstarlet. 2024-09-18T10:30:41.020 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../086-xmlstarlet_1.6.1-2build1_amd64.deb ... 2024-09-18T10:30:41.085 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking xmlstarlet (1.6.1-2build1) ... 2024-09-18T10:30:41.448 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package ceph-test. 2024-09-18T10:30:41.466 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../087-ceph-test_17.2.7-1645-g7061ea6c-1focal_amd64.deb ... 2024-09-18T10:30:41.506 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking ceph-test (17.2.7-1645-g7061ea6c-1focal) ... 2024-09-18T10:30:46.130 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package ceph-volume. 2024-09-18T10:30:46.149 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../088-ceph-volume_17.2.7-1645-g7061ea6c-1focal_all.deb ... 2024-09-18T10:30:46.180 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking ceph-volume (17.2.7-1645-g7061ea6c-1focal) ... 2024-09-18T10:30:46.504 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package fonts-mathjax. 2024-09-18T10:30:46.523 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../089-fonts-mathjax_2.7.4+dfsg-1_all.deb ... 2024-09-18T10:30:46.570 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking fonts-mathjax (2.7.4+dfsg-1) ... 2024-09-18T10:30:47.118 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package libcephfs-dev. 2024-09-18T10:30:47.135 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../090-libcephfs-dev_17.2.7-1645-g7061ea6c-1focal_amd64.deb ... 2024-09-18T10:30:47.193 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking libcephfs-dev (17.2.7-1645-g7061ea6c-1focal) ... 2024-09-18T10:30:47.588 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package nvme-cli. 2024-09-18T10:30:47.608 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../091-nvme-cli_1.9-1ubuntu0.2_amd64.deb ... 2024-09-18T10:30:47.663 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking nvme-cli (1.9-1ubuntu0.2) ... 2024-09-18T10:30:48.034 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package libjs-mathjax. 2024-09-18T10:30:48.051 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../092-libjs-mathjax_2.7.4+dfsg-1_all.deb ... 2024-09-18T10:30:48.092 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking libjs-mathjax (2.7.4+dfsg-1) ... 2024-09-18T10:30:49.781 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package python-natsort-doc. 2024-09-18T10:30:49.799 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../093-python-natsort-doc_7.0.1-1_all.deb ... 2024-09-18T10:30:49.839 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking python-natsort-doc (7.0.1-1) ... 2024-09-18T10:30:50.225 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package python3-atomicwrites. 2024-09-18T10:30:50.245 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../094-python3-atomicwrites_1.1.5-2build1_all.deb ... 2024-09-18T10:30:50.283 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking python3-atomicwrites (1.1.5-2build1) ... 2024-09-18T10:30:50.619 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package python3-attr. 2024-09-18T10:30:50.638 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../095-python3-attr_19.3.0-2_all.deb ... 2024-09-18T10:30:50.677 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking python3-attr (19.3.0-2) ... 2024-09-18T10:30:51.047 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package python3-webencodings. 2024-09-18T10:30:51.066 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../096-python3-webencodings_0.5.1-1ubuntu1_all.deb ... 2024-09-18T10:30:51.130 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking python3-webencodings (0.5.1-1ubuntu1) ... 2024-09-18T10:30:51.466 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package python3-html5lib. 2024-09-18T10:30:51.483 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../097-python3-html5lib_1.0.1-2_all.deb ... 2024-09-18T10:30:51.524 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking python3-html5lib (1.0.1-2) ... 2024-09-18T10:30:51.894 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package python3-lxml:amd64. 2024-09-18T10:30:51.911 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../098-python3-lxml_4.5.0-1ubuntu0.5_amd64.deb ... 2024-09-18T10:30:51.952 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking python3-lxml:amd64 (4.5.0-1ubuntu0.5) ... 2024-09-18T10:30:52.431 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package python3-pastescript. 2024-09-18T10:30:52.447 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../099-python3-pastescript_2.0.2-4_all.deb ... 2024-09-18T10:30:52.489 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2024-09-18T10:30:52.876 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package python3-pluggy. 2024-09-18T10:30:52.892 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../100-python3-pluggy_0.13.0-2_all.deb ... 2024-09-18T10:30:52.959 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking python3-pluggy (0.13.0-2) ... 2024-09-18T10:30:53.304 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package python3-psutil. 2024-09-18T10:30:53.323 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../101-python3-psutil_5.5.1-1ubuntu4_amd64.deb ... 2024-09-18T10:30:53.362 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking python3-psutil (5.5.1-1ubuntu4) ... 2024-09-18T10:30:53.748 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package python3-py. 2024-09-18T10:30:53.767 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../102-python3-py_1.8.1-1ubuntu0.1_all.deb ... 2024-09-18T10:30:53.840 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking python3-py (1.8.1-1ubuntu0.1) ... 2024-09-18T10:30:54.210 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package python3-pyinotify. 2024-09-18T10:30:54.227 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../103-python3-pyinotify_0.9.6-1.2ubuntu1_all.deb ... 2024-09-18T10:30:54.268 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking python3-pyinotify (0.9.6-1.2ubuntu1) ... 2024-09-18T10:30:54.613 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package python3-wcwidth. 2024-09-18T10:30:54.629 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../104-python3-wcwidth_0.1.8+dfsg1-3_all.deb ... 2024-09-18T10:30:54.670 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking python3-wcwidth (0.1.8+dfsg1-3) ... 2024-09-18T10:30:55.057 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package python3-pytest. 2024-09-18T10:30:55.077 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../105-python3-pytest_4.6.9-1_all.deb ... 2024-09-18T10:30:55.115 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking python3-pytest (4.6.9-1) ... 2024-09-18T10:30:55.485 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package python3-simplejson. 2024-09-18T10:30:55.505 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../106-python3-simplejson_3.16.0-2ubuntu2_amd64.deb ... 2024-09-18T10:30:55.543 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking python3-simplejson (3.16.0-2ubuntu2) ... 2024-09-18T10:30:55.854 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package qttranslations5-l10n. 2024-09-18T10:30:55.871 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../107-qttranslations5-l10n_5.12.8-0ubuntu1_all.deb ... 2024-09-18T10:30:55.913 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking qttranslations5-l10n (5.12.8-0ubuntu1) ... 2024-09-18T10:30:56.670 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package radosgw. 2024-09-18T10:30:56.689 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../108-radosgw_17.2.7-1645-g7061ea6c-1focal_amd64.deb ... 2024-09-18T10:30:56.727 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking radosgw (17.2.7-1645-g7061ea6c-1focal) ... 2024-09-18T10:30:58.498 INFO:teuthology.orchestra.run.smithi134.stdout:Selecting previously unselected package rbd-fuse. 2024-09-18T10:30:58.517 INFO:teuthology.orchestra.run.smithi134.stdout:Preparing to unpack .../109-rbd-fuse_17.2.7-1645-g7061ea6c-1focal_amd64.deb ... 2024-09-18T10:30:58.556 INFO:teuthology.orchestra.run.smithi134.stdout:Unpacking rbd-fuse (17.2.7-1645-g7061ea6c-1focal) ... 2024-09-18T10:30:58.931 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up python3-attr (19.3.0-2) ... 2024-09-18T10:30:59.244 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up librabbitmq4:amd64 (0.10.0-1) ... 2024-09-18T10:30:59.379 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up libdouble-conversion3:amd64 (3.1.5-4ubuntu1) ... 2024-09-18T10:30:59.530 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up nvme-cli (1.9-1ubuntu0.2) ... 2024-09-18T10:30:59.723 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up cephadm (17.2.7-1645-g7061ea6c-1focal) ... 2024-09-18T10:31:00.344 INFO:teuthology.orchestra.run.smithi134.stdout:Adding system user cephadm....done 2024-09-18T10:31:00.490 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up fonts-mathjax (2.7.4+dfsg-1) ... 2024-09-18T10:31:00.650 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up python3-waitress (1.4.1-1ubuntu0.1) ... 2024-09-18T10:31:01.001 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up python3-repoze.lru (0.7-2) ... 2024-09-18T10:31:01.303 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up libjs-mathjax (2.7.4+dfsg-1) ... 2024-09-18T10:31:01.437 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up liboath0 (2.6.1-1.3) ... 2024-09-18T10:31:01.588 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up python3-py (1.8.1-1ubuntu0.1) ... 2024-09-18T10:31:01.916 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up python3-joblib (0.14.0-3) ... 2024-09-18T10:31:02.326 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up python3-jwt (1.7.1-2ubuntu2.1) ... 2024-09-18T10:31:02.669 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up python3-cachetools (4.0.0-1) ... 2024-09-18T10:31:03.018 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up python3-bcrypt (3.1.7-2ubuntu1) ... 2024-09-18T10:31:03.330 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up python3-pyinotify (0.9.6-1.2ubuntu1) ... 2024-09-18T10:31:03.665 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up python3-ceph-argparse (17.2.7-1645-g7061ea6c-1focal) ... 2024-09-18T10:31:03.957 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up python3-sklearn-lib (0.22.2.post1+dfsg-5) ... 2024-09-18T10:31:04.083 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up python3-markupsafe (1.1.0-1build2) ... 2024-09-18T10:31:04.393 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up python3-webencodings (0.5.1-1ubuntu1) ... 2024-09-18T10:31:04.719 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up libpcre2-16-0:amd64 (10.34-7ubuntu0.1) ... 2024-09-18T10:31:04.845 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up python3-psutil (5.5.1-1ubuntu4) ... 2024-09-18T10:31:05.213 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up python3-natsort (7.0.1-1) ... 2024-09-18T10:31:05.514 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up python3-atomicwrites (1.1.5-2build1) ... 2024-09-18T10:31:05.799 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up python3-simplejson (3.16.0-2ubuntu2) ... 2024-09-18T10:31:06.150 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up python3-decorator (4.4.2-0ubuntu1) ... 2024-09-18T10:31:06.476 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up python3-jinja2 (2.10.1-2ubuntu0.3) ... 2024-09-18T10:31:06.836 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up python3-tempita (0.5.2-6) ... 2024-09-18T10:31:07.154 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up python3-packaging (20.3-1) ... 2024-09-18T10:31:07.455 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up python-pastedeploy-tpl (2.1.0-1) ... 2024-09-18T10:31:07.581 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up qttranslations5-l10n (5.12.8-0ubuntu1) ... 2024-09-18T10:31:07.732 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up python3-wcwidth (0.1.8+dfsg1-3) ... 2024-09-18T10:31:08.050 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up python3-paste (3.4.0+dfsg1-1) ... 2024-09-18T10:31:08.452 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up python3-werkzeug (0.16.1+dfsg1-2ubuntu0.2) ... 2024-09-18T10:31:08.812 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up python3-cryptography (2.8-3ubuntu0.3) ... 2024-09-18T10:31:09.447 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up socat (1.7.3.3-2) ... 2024-09-18T10:31:09.598 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up python3-ceph-common (17.2.7-1645-g7061ea6c-1focal) ... 2024-09-18T10:31:09.924 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up libqt5core5a:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2024-09-18T10:31:10.084 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up python3-html5lib (1.0.1-2) ... 2024-09-18T10:31:10.418 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up librdkafka1:amd64 (1.2.1-1ubuntu1) ... 2024-09-18T10:31:10.528 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up xmlstarlet (1.6.1-2build1) ... 2024-09-18T10:31:10.629 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up python3-pluggy (0.13.0-2) ... 2024-09-18T10:31:10.955 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up python3-lxml:amd64 (4.5.0-1ubuntu0.5) ... 2024-09-18T10:31:11.307 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up libqt5dbus5:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2024-09-18T10:31:11.433 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up liblua5.3-0:amd64 (5.3.3-1.1ubuntu2) ... 2024-09-18T10:31:11.584 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up python3-pyasn1 (0.4.2-3build1) ... 2024-09-18T10:31:11.927 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up python3-singledispatch (3.4.0.3-2build1) ... 2024-09-18T10:31:12.220 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up python3-logutils (0.3.3-7) ... 2024-09-18T10:31:12.496 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up liblttng-ust0:amd64 (2.11.0-1) ... 2024-09-18T10:31:12.622 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2024-09-18T10:31:12.907 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up python3-prettytable (0.7.2-5) ... 2024-09-18T10:31:13.200 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up python3-soupsieve (1.9.5+dfsg-1) ... 2024-09-18T10:31:13.501 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up python3-websocket (0.53.0-2ubuntu1) ... 2024-09-18T10:31:13.884 INFO:teuthology.orchestra.run.smithi134.stdout:update-alternatives: using /usr/bin/python3-wsdump to provide /usr/bin/wsdump (wsdump) in auto mode 2024-09-18T10:31:14.182 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up libonig5:amd64 (6.9.4-1) ... 2024-09-18T10:31:14.288 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up python3-mako (1.1.0+ds1-1ubuntu2.1) ... 2024-09-18T10:31:14.623 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up python3-cherrypy3 (8.9.1-6) ... 2024-09-18T10:31:14.982 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up libjs-underscore (1.9.1~dfsg-1ubuntu0.20.04.1) ... 2024-09-18T10:31:15.142 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up liblbfgsb0:amd64 (3.0+dfsg.3-7build1) ... 2024-09-18T10:31:15.268 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up python3-webob (1:1.8.5-2ubuntu0.1) ... 2024-09-18T10:31:15.652 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up python3-blinker (1.4+dfsg1-0.3ubuntu1) ... 2024-09-18T10:31:15.970 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up librados2 (17.2.7-1645-g7061ea6c-1focal) ... 2024-09-18T10:31:16.071 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up python3-scipy (1.3.3-3build1) ... 2024-09-18T10:31:17.527 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up libsqlite3-mod-ceph (17.2.7-1645-g7061ea6c-1focal) ... 2024-09-18T10:31:17.645 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up libjq1:amd64 (1.6-1ubuntu0.20.04.1) ... 2024-09-18T10:31:17.780 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up python3-pytest (4.6.9-1) ... 2024-09-18T10:31:18.156 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up python3-openssl (19.0.0-1build1) ... 2024-09-18T10:31:18.466 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up python3-pastedeploy (2.1.0-1) ... 2024-09-18T10:31:18.767 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up libqt5network5:amd64 (5.12.8+dfsg-0ubuntu2.1) ... 2024-09-18T10:31:18.918 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up python3-bs4 (4.8.2-1) ... 2024-09-18T10:31:19.277 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up libcephfs2 (17.2.7-1645-g7061ea6c-1focal) ... 2024-09-18T10:31:19.379 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up python3-oauthlib (3.1.0-1ubuntu2) ... 2024-09-18T10:31:19.681 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up libradosstriper1 (17.2.7-1645-g7061ea6c-1focal) ... 2024-09-18T10:31:19.832 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up python3-pyasn1-modules (0.2.1-0.2build1) ... 2024-09-18T10:31:20.166 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up python3-sklearn (0.22.2.post1+dfsg-5) ... 2024-09-18T10:31:21.249 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up librbd1 (17.2.7-1645-g7061ea6c-1focal) ... 2024-09-18T10:31:21.375 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up libjs-sphinxdoc (1.8.5-7ubuntu3) ... 2024-09-18T10:31:21.501 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up python-natsort-doc (7.0.1-1) ... 2024-09-18T10:31:21.652 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up jq (1.6-1ubuntu0.20.04.1) ... 2024-09-18T10:31:21.778 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up python3-rsa (4.0-3) ... 2024-09-18T10:31:22.087 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up libthrift-0.13.0 (0.13.0-2build2) ... 2024-09-18T10:31:22.230 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up python3-webtest (2.0.32-2) ... 2024-09-18T10:31:22.540 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up ceph-mgr-modules-core (17.2.7-1645-g7061ea6c-1focal) ... 2024-09-18T10:31:22.682 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up ceph-fuse (17.2.7-1645-g7061ea6c-1focal) ... 2024-09-18T10:31:22.939 INFO:teuthology.orchestra.run.smithi134.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-09-18T10:31:22.939 INFO:teuthology.orchestra.run.smithi134.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-09-18T10:31:23.383 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up libcephfs-dev (17.2.7-1645-g7061ea6c-1focal) ... 2024-09-18T10:31:23.620 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up python3-requests-oauthlib (1.0.0-1.1build1) ... 2024-09-18T10:31:23.947 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up python3-pastescript (2.0.2-4) ... 2024-09-18T10:31:24.273 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up python3-pecan (1.3.3-3ubuntu1) ... 2024-09-18T10:31:24.642 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up python3-rados (17.2.7-1645-g7061ea6c-1focal) ... 2024-09-18T10:31:24.768 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up librgw2 (17.2.7-1645-g7061ea6c-1focal) ... 2024-09-18T10:31:24.919 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up python3-rbd (17.2.7-1645-g7061ea6c-1focal) ... 2024-09-18T10:31:25.045 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up rbd-fuse (17.2.7-1645-g7061ea6c-1focal) ... 2024-09-18T10:31:25.204 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up python3-routes (2.4.1-2) ... 2024-09-18T10:31:25.506 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up python3-google-auth (1.5.1-2) ... 2024-09-18T10:31:25.808 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up python3-rgw (17.2.7-1645-g7061ea6c-1focal) ... 2024-09-18T10:31:25.933 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up python3-cephfs (17.2.7-1645-g7061ea6c-1focal) ... 2024-09-18T10:31:26.093 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up python3-kubernetes (7.0.0~a1-2) ... 2024-09-18T10:31:27.013 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up ceph-common (17.2.7-1645-g7061ea6c-1focal) ... 2024-09-18T10:31:27.573 INFO:teuthology.orchestra.run.smithi134.stdout:Adding group ceph....done 2024-09-18T10:31:27.992 INFO:teuthology.orchestra.run.smithi134.stdout:Adding system user ceph....done 2024-09-18T10:31:28.168 INFO:teuthology.orchestra.run.smithi134.stdout:Setting system user ceph properties....done 2024-09-18T10:31:28.192 INFO:teuthology.orchestra.run.smithi134.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2024-09-18T10:31:28.342 INFO:teuthology.orchestra.run.smithi134.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2024-09-18T10:31:28.648 INFO:teuthology.orchestra.run.smithi134.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2024-09-18T10:31:29.114 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up ceph-test (17.2.7-1645-g7061ea6c-1focal) ... 2024-09-18T10:31:29.240 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up radosgw (17.2.7-1645-g7061ea6c-1focal) ... 2024-09-18T10:31:29.529 INFO:teuthology.orchestra.run.smithi134.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-09-18T10:31:29.529 INFO:teuthology.orchestra.run.smithi134.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-09-18T10:31:30.159 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up ceph-base (17.2.7-1645-g7061ea6c-1focal) ... 2024-09-18T10:31:30.573 INFO:teuthology.orchestra.run.smithi134.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2024-09-18T10:31:30.985 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up ceph-mds (17.2.7-1645-g7061ea6c-1focal) ... 2024-09-18T10:31:31.209 INFO:teuthology.orchestra.run.smithi134.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-09-18T10:31:31.210 INFO:teuthology.orchestra.run.smithi134.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-09-18T10:31:31.637 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up ceph-mgr (17.2.7-1645-g7061ea6c-1focal) ... 2024-09-18T10:31:31.860 INFO:teuthology.orchestra.run.smithi134.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-09-18T10:31:31.860 INFO:teuthology.orchestra.run.smithi134.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-09-18T10:31:32.288 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up ceph-osd (17.2.7-1645-g7061ea6c-1focal) ... 2024-09-18T10:31:32.798 INFO:teuthology.orchestra.run.smithi134.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-09-18T10:31:32.798 INFO:teuthology.orchestra.run.smithi134.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-09-18T10:31:33.265 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up ceph-mgr-k8sevents (17.2.7-1645-g7061ea6c-1focal) ... 2024-09-18T10:31:33.391 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up ceph-mgr-diskprediction-local (17.2.7-1645-g7061ea6c-1focal) ... 2024-09-18T10:31:33.567 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up ceph-mon (17.2.7-1645-g7061ea6c-1focal) ... 2024-09-18T10:31:33.795 INFO:teuthology.orchestra.run.smithi134.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-09-18T10:31:33.795 INFO:teuthology.orchestra.run.smithi134.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-09-18T10:31:34.235 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up ceph-mgr-cephadm (17.2.7-1645-g7061ea6c-1focal) ... 2024-09-18T10:31:34.377 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up ceph (17.2.7-1645-g7061ea6c-1focal) ... 2024-09-18T10:31:34.537 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up ceph-mgr-dashboard (17.2.7-1645-g7061ea6c-1focal) ... 2024-09-18T10:31:34.680 INFO:teuthology.orchestra.run.smithi134.stdout:Setting up ceph-volume (17.2.7-1645-g7061ea6c-1focal) ... 2024-09-18T10:31:34.982 INFO:teuthology.orchestra.run.smithi134.stdout:Processing triggers for libc-bin (2.31-0ubuntu9.9) ... 2024-09-18T10:31:35.159 INFO:teuthology.orchestra.run.smithi134.stdout:Processing triggers for systemd (245.4-4ubuntu3.17) ... 2024-09-18T10:31:35.452 INFO:teuthology.orchestra.run.smithi134.stdout:Processing triggers for man-db (2.9.1-1) ... 2024-09-18T10:31:38.250 INFO:teuthology.orchestra.run.smithi134.stdout:Processing triggers for fontconfig (2.13.1-2ubuntu3) ... 2024-09-18T10:31:38.873 INFO:teuthology.orchestra.run.smithi134.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-09-18T10:31:38.879 DEBUG:teuthology.parallel:result is None 2024-09-18T10:31:38.879 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F20.04%2Fx86_64&sha1=7061ea6ca50908d104e9f3979dc615e118d9e153 2024-09-18T10:31:39.024 DEBUG:teuthology.orchestra.run.smithi134:> dpkg-query -W -f '${Version}' ceph 2024-09-18T10:31:39.049 INFO:teuthology.orchestra.run.smithi134.stdout:17.2.7-1645-g7061ea6c-1focal 2024-09-18T10:31:39.049 INFO:teuthology.packaging:The installed version of ceph is 17.2.7-1645-g7061ea6c-1focal 2024-09-18T10:31:39.050 INFO:teuthology.task.install:The correct ceph version 17.2.7-1645-g7061ea6c-1focal is installed. 2024-09-18T10:31:39.052 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-09-18T10:31:39.053 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-09-18T10:31:39.053 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-18T10:31:39.106 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-09-18T10:31:39.106 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-09-18T10:31:39.106 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd of=/usr/bin/daemon-helper 2024-09-18T10:31:39.118 DEBUG:teuthology.orchestra.run.smithi134:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-18T10:31:39.129 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-09-18T10:31:39.129 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-09-18T10:31:39.130 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-18T10:31:39.185 DEBUG:teuthology.orchestra.run.smithi134:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-18T10:31:39.197 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-09-18T10:31:39.197 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-09-18T10:31:39.197 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd of=/usr/bin/stdin-killer 2024-09-18T10:31:39.211 DEBUG:teuthology.orchestra.run.smithi134:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-18T10:31:39.267 INFO:teuthology.run_tasks:Running task ceph... 2024-09-18T10:31:39.347 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-09-18T10:31:39.348 DEBUG:teuthology.orchestra.run.smithi134:> sudo chmod 777 /var/log/ceph 2024-09-18T10:31:39.363 INFO:tasks.ceph:Disabling ceph logrotate... 2024-09-18T10:31:39.364 DEBUG:teuthology.orchestra.run.smithi134:> sudo rm -f -- /etc/logrotate.d/ceph 2024-09-18T10:31:39.425 INFO:tasks.ceph:Creating extra log directories... 2024-09-18T10:31:39.426 DEBUG:teuthology.orchestra.run.smithi134:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-09-18T10:31:39.443 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-09-18T10:31:39.443 INFO:tasks.ceph:config {'conf': {'osd': {'osd max object name len': 400, 'osd max object namespace len': 64, 'bluestore zero block detection': True, 'debug ms': 1, 'debug osd': 20, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd mclock override recovery settings': True, 'osd mclock profile': 'high_recovery_ops', 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random'}, 'global': {'mon election default strategy': 1}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon scrub interval': 300}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['but it is still running', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_APP_NOT_ENABLED\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-09-18T10:31:39.444 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/skanta-2024-09-18_09:13:00-rados-wip-bharath11-testing-2024-09-18-1115-quincy-distro-default-smithi/7910878', 'branch': 'wip-bharath11-testing-2024-09-18-1115-quincy', 'description': 'rados/singleton-nomsgr/{all/health-warnings mon_election/classic rados supported-random-distro$/{ubuntu_latest}}', 'email': 'skanta@redhat.com', 'first_in_suite': False, 'job_id': '7910878', 'kernel': {'kdb': 1, 'sha1': 'distro'}, 'ktype': 'distro', 'last_in_suite': False, 'machine_type': 'smithi', 'name': 'skanta-2024-09-18_09:13:00-rados-wip-bharath11-testing-2024-09-18-1115-quincy-distro-default-smithi', 'no_nested_subset': False, 'os_type': 'ubuntu', 'os_version': '20.04', 'overrides': {'admin_socket': {'branch': 'wip-bharath11-testing-2024-09-18-1115-quincy'}, 'ceph': {'conf': {'global': {'mon election default strategy': 1}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon scrub interval': 300}, 'osd': {'bluestore zero block detection': True, 'debug ms': 1, 'debug osd': 20, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd mclock override recovery settings': True, 'osd mclock profile': 'high_recovery_ops', 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random'}}, 'flavor': 'default', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)'], 'sha1': '7061ea6ca50908d104e9f3979dc615e118d9e153'}, 'ceph-deploy': {'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}}}, 'install': {'ceph': {'flavor': 'default', 'sha1': '7061ea6ca50908d104e9f3979dc615e118d9e153'}}, 'workunit': {'branch': 'wip-bharath11-testing-2024-09-18-1115-quincy', 'sha1': '7061ea6ca50908d104e9f3979dc615e118d9e153'}}, 'owner': 'scheduled_skanta@teuthology', 'priority': 99, 'repo': 'https://github.com/ceph/ceph-ci.git', 'roles': [['mon.a', 'mgr.x', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'osd.8', 'osd.9', 'client.0']], 'seed': 4631, 'sha1': '7061ea6ca50908d104e9f3979dc615e118d9e153', 'sleep_before_teardown': 0, 'subset': '111/120000', 'suite': 'rados', 'suite_branch': 'wip-bharath11-testing-2024-09-18-1115-quincy', 'suite_path': '/home/teuthworker/src/github.com_ceph_ceph-c_7061ea6ca50908d104e9f3979dc615e118d9e153/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://github.com/ceph/ceph-ci.git', 'suite_sha1': '7061ea6ca50908d104e9f3979dc615e118d9e153', 'targets': {'smithi134.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFyT/r7xuKfmg50w0OdQebr4A/ptJo1Zk6DyZNZNZM/q5rW+sznh+wvHqeT3e4YtCdZLaXyvMms8o8u8H5nXOYM='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'kdb': 1, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': None}, {'ceph': {'conf': {'osd': {'osd max object name len': 400, 'osd max object namespace len': 64, 'bluestore zero block detection': True, 'debug ms': 1, 'debug osd': 20, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd mclock override recovery settings': True, 'osd mclock profile': 'high_recovery_ops', 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random'}, 'global': {'mon election default strategy': 1}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon scrub interval': 300}}, 'log-ignorelist': ['but it is still running', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_APP_NOT_ENABLED\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)'], 'pre-mgr-commands': ['sudo ceph config set mgr mgr_pool false --force'], 'flavor': 'default', 'sha1': '7061ea6ca50908d104e9f3979dc615e118d9e153', 'cluster': 'ceph'}}, {'workunit': {'clients': {'all': ['rados/test_health_warnings.sh']}}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': []}, 'teuthology_branch': 'main', 'teuthology_sha1': '3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a', 'timestamp': '2024-09-18_09:13:00', 'tube': 'smithi', 'user': 'skanta', 'verbose': True, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856'} 2024-09-18T10:31:39.444 DEBUG:teuthology.orchestra.run.smithi134:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-09-18T10:31:39.450 DEBUG:teuthology.orchestra.run.smithi134:> sudo install -d -m0777 -- /var/run/ceph 2024-09-18T10:31:39.505 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-09-18T10:31:39.505 DEBUG:teuthology.orchestra.run.smithi134:> dd if=/scratch_devs of=/dev/stdout 2024-09-18T10:31:39.512 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-18T10:31:39.512 DEBUG:teuthology.orchestra.run.smithi134:> stat /dev/vg_nvme/lv_1 2024-09-18T10:31:39.518 INFO:teuthology.orchestra.run.smithi134.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-18T10:31:39.519 INFO:teuthology.orchestra.run.smithi134.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-18T10:31:39.519 INFO:teuthology.orchestra.run.smithi134.stdout:Device: 6h/6d Inode: 361 Links: 1 2024-09-18T10:31:39.520 INFO:teuthology.orchestra.run.smithi134.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-18T10:31:39.521 INFO:teuthology.orchestra.run.smithi134.stdout:Access: 2024-09-18 10:28:10.292813268 +0000 2024-09-18T10:31:39.521 INFO:teuthology.orchestra.run.smithi134.stdout:Modify: 2024-09-18 10:28:10.068818435 +0000 2024-09-18T10:31:39.521 INFO:teuthology.orchestra.run.smithi134.stdout:Change: 2024-09-18 10:28:10.068818435 +0000 2024-09-18T10:31:39.521 INFO:teuthology.orchestra.run.smithi134.stdout: Birth: - 2024-09-18T10:31:39.521 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-18T10:31:39.534 INFO:teuthology.orchestra.run.smithi134.stderr:1+0 records in 2024-09-18T10:31:39.534 INFO:teuthology.orchestra.run.smithi134.stderr:1+0 records out 2024-09-18T10:31:39.534 INFO:teuthology.orchestra.run.smithi134.stderr:512 bytes copied, 0.000397119 s, 1.3 MB/s 2024-09-18T10:31:39.536 DEBUG:teuthology.orchestra.run.smithi134:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-18T10:31:39.587 DEBUG:teuthology.orchestra.run.smithi134:> stat /dev/vg_nvme/lv_2 2024-09-18T10:31:39.595 INFO:teuthology.orchestra.run.smithi134.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-18T10:31:39.595 INFO:teuthology.orchestra.run.smithi134.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-18T10:31:39.595 INFO:teuthology.orchestra.run.smithi134.stdout:Device: 6h/6d Inode: 367 Links: 1 2024-09-18T10:31:39.595 INFO:teuthology.orchestra.run.smithi134.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-18T10:31:39.595 INFO:teuthology.orchestra.run.smithi134.stdout:Access: 2024-09-18 10:28:10.516808103 +0000 2024-09-18T10:31:39.595 INFO:teuthology.orchestra.run.smithi134.stdout:Modify: 2024-09-18 10:28:10.512808195 +0000 2024-09-18T10:31:39.595 INFO:teuthology.orchestra.run.smithi134.stdout:Change: 2024-09-18 10:28:10.512808195 +0000 2024-09-18T10:31:39.595 INFO:teuthology.orchestra.run.smithi134.stdout: Birth: - 2024-09-18T10:31:39.596 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-18T10:31:39.609 INFO:teuthology.orchestra.run.smithi134.stderr:1+0 records in 2024-09-18T10:31:39.609 INFO:teuthology.orchestra.run.smithi134.stderr:1+0 records out 2024-09-18T10:31:39.609 INFO:teuthology.orchestra.run.smithi134.stderr:512 bytes copied, 0.000394152 s, 1.3 MB/s 2024-09-18T10:31:39.610 DEBUG:teuthology.orchestra.run.smithi134:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-18T10:31:39.663 DEBUG:teuthology.orchestra.run.smithi134:> stat /dev/vg_nvme/lv_3 2024-09-18T10:31:39.669 INFO:teuthology.orchestra.run.smithi134.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-18T10:31:39.670 INFO:teuthology.orchestra.run.smithi134.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-18T10:31:39.670 INFO:teuthology.orchestra.run.smithi134.stdout:Device: 6h/6d Inode: 371 Links: 1 2024-09-18T10:31:39.670 INFO:teuthology.orchestra.run.smithi134.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-18T10:31:39.670 INFO:teuthology.orchestra.run.smithi134.stdout:Access: 2024-09-18 10:28:11.152793436 +0000 2024-09-18T10:31:39.670 INFO:teuthology.orchestra.run.smithi134.stdout:Modify: 2024-09-18 10:28:10.960797864 +0000 2024-09-18T10:31:39.670 INFO:teuthology.orchestra.run.smithi134.stdout:Change: 2024-09-18 10:28:10.960797864 +0000 2024-09-18T10:31:39.670 INFO:teuthology.orchestra.run.smithi134.stdout: Birth: - 2024-09-18T10:31:39.670 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-18T10:31:39.684 INFO:teuthology.orchestra.run.smithi134.stderr:1+0 records in 2024-09-18T10:31:39.684 INFO:teuthology.orchestra.run.smithi134.stderr:1+0 records out 2024-09-18T10:31:39.685 INFO:teuthology.orchestra.run.smithi134.stderr:512 bytes copied, 0.000404634 s, 1.3 MB/s 2024-09-18T10:31:39.685 DEBUG:teuthology.orchestra.run.smithi134:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-18T10:31:39.735 DEBUG:teuthology.orchestra.run.smithi134:> stat /dev/vg_nvme/lv_4 2024-09-18T10:31:39.742 INFO:teuthology.orchestra.run.smithi134.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-18T10:31:39.742 INFO:teuthology.orchestra.run.smithi134.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-18T10:31:39.742 INFO:teuthology.orchestra.run.smithi134.stdout:Device: 6h/6d Inode: 377 Links: 1 2024-09-18T10:31:39.742 INFO:teuthology.orchestra.run.smithi134.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-18T10:31:39.742 INFO:teuthology.orchestra.run.smithi134.stdout:Access: 2024-09-18 10:28:11.588783382 +0000 2024-09-18T10:31:39.743 INFO:teuthology.orchestra.run.smithi134.stdout:Modify: 2024-09-18 10:28:11.376788270 +0000 2024-09-18T10:31:39.743 INFO:teuthology.orchestra.run.smithi134.stdout:Change: 2024-09-18 10:28:11.376788270 +0000 2024-09-18T10:31:39.743 INFO:teuthology.orchestra.run.smithi134.stdout: Birth: - 2024-09-18T10:31:39.743 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-18T10:31:39.755 INFO:teuthology.orchestra.run.smithi134.stderr:1+0 records in 2024-09-18T10:31:39.755 INFO:teuthology.orchestra.run.smithi134.stderr:1+0 records out 2024-09-18T10:31:39.756 INFO:teuthology.orchestra.run.smithi134.stderr:512 bytes copied, 0.000371499 s, 1.4 MB/s 2024-09-18T10:31:39.757 DEBUG:teuthology.orchestra.run.smithi134:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-18T10:31:39.764 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-18T10:31:39.765 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi134.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}} 2024-09-18T10:31:39.765 INFO:tasks.ceph:Generating config... 2024-09-18T10:31:39.766 INFO:tasks.ceph:[osd] osd max object name len = 400 2024-09-18T10:31:39.766 INFO:tasks.ceph:[osd] osd max object namespace len = 64 2024-09-18T10:31:39.766 INFO:tasks.ceph:[osd] bluestore zero block detection = True 2024-09-18T10:31:39.766 INFO:tasks.ceph:[osd] debug ms = 1 2024-09-18T10:31:39.766 INFO:tasks.ceph:[osd] debug osd = 20 2024-09-18T10:31:39.766 INFO:tasks.ceph:[osd] osd debug verify cached snaps = True 2024-09-18T10:31:39.766 INFO:tasks.ceph:[osd] osd debug verify missing on start = True 2024-09-18T10:31:39.766 INFO:tasks.ceph:[osd] osd mclock override recovery settings = True 2024-09-18T10:31:39.766 INFO:tasks.ceph:[osd] osd mclock profile = high_recovery_ops 2024-09-18T10:31:39.767 INFO:tasks.ceph:[osd] osd op queue = debug_random 2024-09-18T10:31:39.767 INFO:tasks.ceph:[osd] osd op queue cut off = debug_random 2024-09-18T10:31:39.767 INFO:tasks.ceph:[global] mon election default strategy = 1 2024-09-18T10:31:39.767 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-09-18T10:31:39.767 INFO:tasks.ceph:[mgr] debug ms = 1 2024-09-18T10:31:39.767 INFO:tasks.ceph:[mon] debug mon = 20 2024-09-18T10:31:39.767 INFO:tasks.ceph:[mon] debug ms = 1 2024-09-18T10:31:39.767 INFO:tasks.ceph:[mon] debug paxos = 20 2024-09-18T10:31:39.767 INFO:tasks.ceph:[mon] mon scrub interval = 300 2024-09-18T10:31:39.767 INFO:tasks.ceph:Setting up mon.a... 2024-09-18T10:31:39.767 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-09-18T10:31:39.799 INFO:teuthology.orchestra.run.smithi134.stdout:creating /etc/ceph/ceph.keyring 2024-09-18T10:31:39.801 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-09-18T10:31:39.878 DEBUG:teuthology.orchestra.run.smithi134:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-18T10:31:39.893 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.134')] 2024-09-18T10:31:39.893 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.134', 'mon election default strategy': 1}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'osd max object name len': 400, 'osd max object namespace len': 64, 'bluestore zero block detection': True, 'debug ms': 1, 'debug osd': 20, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd mclock override recovery settings': True, 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random'}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon scrub interval': 300}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok'}, 'mon.a': {}} 2024-09-18T10:31:39.894 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-09-18T10:31:39.894 DEBUG:teuthology.orchestra.run.smithi134:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-09-18T10:31:39.900 DEBUG:teuthology.orchestra.run.smithi134:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.134 --print /home/ubuntu/cephtest/ceph.monmap 2024-09-18T10:31:39.976 INFO:teuthology.orchestra.run.smithi134.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-09-18T10:31:39.977 INFO:teuthology.orchestra.run.smithi134.stdout:monmaptool: generated fsid 909a0a21-79ad-49e6-bd09-2f76d1ea12ab 2024-09-18T10:31:39.977 INFO:teuthology.orchestra.run.smithi134.stdout:setting min_mon_release = octopus 2024-09-18T10:31:39.977 INFO:teuthology.orchestra.run.smithi134.stdout:epoch 0 2024-09-18T10:31:39.977 INFO:teuthology.orchestra.run.smithi134.stdout:fsid 909a0a21-79ad-49e6-bd09-2f76d1ea12ab 2024-09-18T10:31:39.977 INFO:teuthology.orchestra.run.smithi134.stdout:last_changed 2024-09-18T10:31:39.977519+0000 2024-09-18T10:31:39.977 INFO:teuthology.orchestra.run.smithi134.stdout:created 2024-09-18T10:31:39.977519+0000 2024-09-18T10:31:39.977 INFO:teuthology.orchestra.run.smithi134.stdout:min_mon_release 15 (octopus) 2024-09-18T10:31:39.977 INFO:teuthology.orchestra.run.smithi134.stdout:election_strategy: 1 2024-09-18T10:31:39.977 INFO:teuthology.orchestra.run.smithi134.stdout:0: [v2:172.21.15.134:3300/0,v1:172.21.15.134:6789/0] mon.a 2024-09-18T10:31:39.977 INFO:teuthology.orchestra.run.smithi134.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (1 monitors) 2024-09-18T10:31:39.979 DEBUG:teuthology.orchestra.run.smithi134:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-09-18T10:31:39.984 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 909a0a21-79ad-49e6-bd09-2f76d1ea12ab... 2024-09-18T10:31:39.985 DEBUG:teuthology.orchestra.run.smithi134:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-18T10:31:40.020 INFO:teuthology.orchestra.run.smithi134.stdout:[global] 2024-09-18T10:31:40.020 INFO:teuthology.orchestra.run.smithi134.stdout: chdir = "" 2024-09-18T10:31:40.020 INFO:teuthology.orchestra.run.smithi134.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-18T10:31:40.020 INFO:teuthology.orchestra.run.smithi134.stdout: auth supported = cephx 2024-09-18T10:31:40.021 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-18T10:31:40.021 INFO:teuthology.orchestra.run.smithi134.stdout: filestore xattr use omap = true 2024-09-18T10:31:40.021 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-18T10:31:40.021 INFO:teuthology.orchestra.run.smithi134.stdout: mon clock drift allowed = 1.000 2024-09-18T10:31:40.021 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-18T10:31:40.022 INFO:teuthology.orchestra.run.smithi134.stdout: osd crush chooseleaf type = 0 2024-09-18T10:31:40.022 INFO:teuthology.orchestra.run.smithi134.stdout: auth debug = true 2024-09-18T10:31:40.022 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-18T10:31:40.022 INFO:teuthology.orchestra.run.smithi134.stdout: ms die on old message = true 2024-09-18T10:31:40.022 INFO:teuthology.orchestra.run.smithi134.stdout: ms die on bug = true 2024-09-18T10:31:40.023 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-18T10:31:40.023 INFO:teuthology.orchestra.run.smithi134.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-18T10:31:40.023 INFO:teuthology.orchestra.run.smithi134.stdout: mon pg warn max object skew = 0 2024-09-18T10:31:40.023 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-18T10:31:40.023 INFO:teuthology.orchestra.run.smithi134.stdout: # disable pg_autoscaler by default for new pools 2024-09-18T10:31:40.023 INFO:teuthology.orchestra.run.smithi134.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-18T10:31:40.024 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-18T10:31:40.024 INFO:teuthology.orchestra.run.smithi134.stdout: osd pool default size = 2 2024-09-18T10:31:40.024 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-18T10:31:40.024 INFO:teuthology.orchestra.run.smithi134.stdout: mon osd allow primary affinity = true 2024-09-18T10:31:40.024 INFO:teuthology.orchestra.run.smithi134.stdout: mon osd allow pg remap = true 2024-09-18T10:31:40.025 INFO:teuthology.orchestra.run.smithi134.stdout: mon warn on legacy crush tunables = false 2024-09-18T10:31:40.025 INFO:teuthology.orchestra.run.smithi134.stdout: mon warn on crush straw calc version zero = false 2024-09-18T10:31:40.025 INFO:teuthology.orchestra.run.smithi134.stdout: mon warn on no sortbitwise = false 2024-09-18T10:31:40.025 INFO:teuthology.orchestra.run.smithi134.stdout: mon warn on osd down out interval zero = false 2024-09-18T10:31:40.025 INFO:teuthology.orchestra.run.smithi134.stdout: mon warn on too few osds = false 2024-09-18T10:31:40.025 INFO:teuthology.orchestra.run.smithi134.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-18T10:31:40.026 INFO:teuthology.orchestra.run.smithi134.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-18T10:31:40.026 INFO:teuthology.orchestra.run.smithi134.stdout: mon_allow_pool_size_one = true 2024-09-18T10:31:40.026 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-18T10:31:40.026 INFO:teuthology.orchestra.run.smithi134.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-18T10:31:40.026 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-18T10:31:40.027 INFO:teuthology.orchestra.run.smithi134.stdout: osd default data pool replay window = 5 2024-09-18T10:31:40.027 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-18T10:31:40.027 INFO:teuthology.orchestra.run.smithi134.stdout: mon allow pool delete = true 2024-09-18T10:31:40.027 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-18T10:31:40.027 INFO:teuthology.orchestra.run.smithi134.stdout: mon cluster log file level = debug 2024-09-18T10:31:40.027 INFO:teuthology.orchestra.run.smithi134.stdout: debug asserts on shutdown = true 2024-09-18T10:31:40.028 INFO:teuthology.orchestra.run.smithi134.stdout: mon health detail to clog = false 2024-09-18T10:31:40.028 INFO:teuthology.orchestra.run.smithi134.stdout: mon host = 172.21.15.134 2024-09-18T10:31:40.028 INFO:teuthology.orchestra.run.smithi134.stdout: mon election default strategy = 1 2024-09-18T10:31:40.028 INFO:teuthology.orchestra.run.smithi134.stdout: fsid = 909a0a21-79ad-49e6-bd09-2f76d1ea12ab 2024-09-18T10:31:40.028 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-18T10:31:40.028 INFO:teuthology.orchestra.run.smithi134.stdout:[osd] 2024-09-18T10:31:40.029 INFO:teuthology.orchestra.run.smithi134.stdout: osd journal size = 100 2024-09-18T10:31:40.029 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-18T10:31:40.029 INFO:teuthology.orchestra.run.smithi134.stdout: osd scrub load threshold = 5.0 2024-09-18T10:31:40.029 INFO:teuthology.orchestra.run.smithi134.stdout: osd scrub max interval = 600 2024-09-18T10:31:40.029 INFO:teuthology.orchestra.run.smithi134.stdout: osd mclock profile = high_recovery_ops 2024-09-18T10:31:40.029 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-18T10:31:40.030 INFO:teuthology.orchestra.run.smithi134.stdout: osd recover clone overlap = true 2024-09-18T10:31:40.030 INFO:teuthology.orchestra.run.smithi134.stdout: osd recovery max chunk = 1048576 2024-09-18T10:31:40.030 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-18T10:31:40.030 INFO:teuthology.orchestra.run.smithi134.stdout: osd debug shutdown = true 2024-09-18T10:31:40.030 INFO:teuthology.orchestra.run.smithi134.stdout: osd debug op order = true 2024-09-18T10:31:40.031 INFO:teuthology.orchestra.run.smithi134.stdout: osd debug verify stray on activate = true 2024-09-18T10:31:40.031 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-18T10:31:40.031 INFO:teuthology.orchestra.run.smithi134.stdout: osd open classes on start = true 2024-09-18T10:31:40.031 INFO:teuthology.orchestra.run.smithi134.stdout: osd debug pg log writeout = true 2024-09-18T10:31:40.031 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-18T10:31:40.031 INFO:teuthology.orchestra.run.smithi134.stdout: osd deep scrub update digest min age = 30 2024-09-18T10:31:40.032 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-18T10:31:40.032 INFO:teuthology.orchestra.run.smithi134.stdout: osd map max advance = 10 2024-09-18T10:31:40.032 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-18T10:31:40.032 INFO:teuthology.orchestra.run.smithi134.stdout: journal zero on create = true 2024-09-18T10:31:40.032 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-18T10:31:40.032 INFO:teuthology.orchestra.run.smithi134.stdout: filestore ondisk finisher threads = 3 2024-09-18T10:31:40.032 INFO:teuthology.orchestra.run.smithi134.stdout: filestore apply finisher threads = 3 2024-09-18T10:31:40.032 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-18T10:31:40.032 INFO:teuthology.orchestra.run.smithi134.stdout: bdev debug aio = true 2024-09-18T10:31:40.032 INFO:teuthology.orchestra.run.smithi134.stdout: osd debug misdirected ops = true 2024-09-18T10:31:40.032 INFO:teuthology.orchestra.run.smithi134.stdout: osd max object name len = 400 2024-09-18T10:31:40.032 INFO:teuthology.orchestra.run.smithi134.stdout: osd max object namespace len = 64 2024-09-18T10:31:40.033 INFO:teuthology.orchestra.run.smithi134.stdout: bluestore zero block detection = True 2024-09-18T10:31:40.033 INFO:teuthology.orchestra.run.smithi134.stdout: debug ms = 1 2024-09-18T10:31:40.033 INFO:teuthology.orchestra.run.smithi134.stdout: debug osd = 20 2024-09-18T10:31:40.033 INFO:teuthology.orchestra.run.smithi134.stdout: osd debug verify cached snaps = True 2024-09-18T10:31:40.033 INFO:teuthology.orchestra.run.smithi134.stdout: osd debug verify missing on start = True 2024-09-18T10:31:40.033 INFO:teuthology.orchestra.run.smithi134.stdout: osd mclock override recovery settings = True 2024-09-18T10:31:40.033 INFO:teuthology.orchestra.run.smithi134.stdout: osd op queue = debug_random 2024-09-18T10:31:40.033 INFO:teuthology.orchestra.run.smithi134.stdout: osd op queue cut off = debug_random 2024-09-18T10:31:40.033 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-18T10:31:40.034 INFO:teuthology.orchestra.run.smithi134.stdout:[mgr] 2024-09-18T10:31:40.034 INFO:teuthology.orchestra.run.smithi134.stdout: debug ms = 1 2024-09-18T10:31:40.034 INFO:teuthology.orchestra.run.smithi134.stdout: debug mgr = 20 2024-09-18T10:31:40.034 INFO:teuthology.orchestra.run.smithi134.stdout: debug mon = 20 2024-09-18T10:31:40.034 INFO:teuthology.orchestra.run.smithi134.stdout: debug auth = 20 2024-09-18T10:31:40.034 INFO:teuthology.orchestra.run.smithi134.stdout: mon reweight min pgs per osd = 4 2024-09-18T10:31:40.034 INFO:teuthology.orchestra.run.smithi134.stdout: mon reweight min bytes per osd = 10 2024-09-18T10:31:40.034 INFO:teuthology.orchestra.run.smithi134.stdout: mgr/telemetry/nag = false 2024-09-18T10:31:40.034 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-18T10:31:40.035 INFO:teuthology.orchestra.run.smithi134.stdout:[mon] 2024-09-18T10:31:40.035 INFO:teuthology.orchestra.run.smithi134.stdout: debug ms = 1 2024-09-18T10:31:40.035 INFO:teuthology.orchestra.run.smithi134.stdout: debug mon = 20 2024-09-18T10:31:40.035 INFO:teuthology.orchestra.run.smithi134.stdout: debug paxos = 20 2024-09-18T10:31:40.035 INFO:teuthology.orchestra.run.smithi134.stdout: debug auth = 20 2024-09-18T10:31:40.035 INFO:teuthology.orchestra.run.smithi134.stdout: mon data avail warn = 5 2024-09-18T10:31:40.035 INFO:teuthology.orchestra.run.smithi134.stdout: mon mgr mkfs grace = 240 2024-09-18T10:31:40.036 INFO:teuthology.orchestra.run.smithi134.stdout: mon reweight min pgs per osd = 4 2024-09-18T10:31:40.036 INFO:teuthology.orchestra.run.smithi134.stdout: mon osd reporter subtree level = osd 2024-09-18T10:31:40.036 INFO:teuthology.orchestra.run.smithi134.stdout: mon osd prime pg temp = true 2024-09-18T10:31:40.036 INFO:teuthology.orchestra.run.smithi134.stdout: mon reweight min bytes per osd = 10 2024-09-18T10:31:40.036 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-18T10:31:40.036 INFO:teuthology.orchestra.run.smithi134.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-18T10:31:40.037 INFO:teuthology.orchestra.run.smithi134.stdout: auth mon ticket ttl = 660 # 11m 2024-09-18T10:31:40.037 INFO:teuthology.orchestra.run.smithi134.stdout: auth service ticket ttl = 240 # 4m 2024-09-18T10:31:40.037 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-18T10:31:40.037 INFO:teuthology.orchestra.run.smithi134.stdout: # don't complain about insecure global_id in the test suite 2024-09-18T10:31:40.037 INFO:teuthology.orchestra.run.smithi134.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-18T10:31:40.037 INFO:teuthology.orchestra.run.smithi134.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-18T10:31:40.037 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-18T10:31:40.037 INFO:teuthology.orchestra.run.smithi134.stdout: # 1m isn't quite enough 2024-09-18T10:31:40.037 INFO:teuthology.orchestra.run.smithi134.stdout: mon_down_mkfs_grace = 2m 2024-09-18T10:31:40.037 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-18T10:31:40.038 INFO:teuthology.orchestra.run.smithi134.stdout: mon_warn_on_filestore_osds = false 2024-09-18T10:31:40.038 INFO:teuthology.orchestra.run.smithi134.stdout: mon scrub interval = 300 2024-09-18T10:31:40.038 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-18T10:31:40.038 INFO:teuthology.orchestra.run.smithi134.stdout:[client] 2024-09-18T10:31:40.038 INFO:teuthology.orchestra.run.smithi134.stdout: rgw cache enabled = true 2024-09-18T10:31:40.038 INFO:teuthology.orchestra.run.smithi134.stdout: rgw enable ops log = true 2024-09-18T10:31:40.039 INFO:teuthology.orchestra.run.smithi134.stdout: rgw enable usage log = true 2024-09-18T10:31:40.039 INFO:teuthology.orchestra.run.smithi134.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-18T10:31:40.039 INFO:teuthology.orchestra.run.smithi134.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-18T10:31:40.039 INFO:teuthology.orchestra.run.smithi134.stdout:[mon.a] 2024-09-18T10:31:40.040 INFO:tasks.ceph:Creating admin key on mon.a... 2024-09-18T10:31:40.040 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-09-18T10:31:40.135 INFO:tasks.ceph:Copying monmap to all nodes... 2024-09-18T10:31:40.135 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-09-18T10:31:40.135 DEBUG:teuthology.orchestra.run.smithi134:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-09-18T10:31:40.142 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-09-18T10:31:40.142 DEBUG:teuthology.orchestra.run.smithi134:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-09-18T10:31:40.148 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi134.front.sepia.ceph.com 2024-09-18T10:31:40.148 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-09-18T10:31:40.148 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd of=/etc/ceph/ceph.keyring 2024-09-18T10:31:40.148 DEBUG:teuthology.orchestra.run.smithi134:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-18T10:31:40.172 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-09-18T10:31:40.172 DEBUG:teuthology.orchestra.run.smithi134:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-09-18T10:31:40.234 INFO:tasks.ceph:Setting up mon nodes... 2024-09-18T10:31:40.234 INFO:tasks.ceph:Setting up mgr nodes... 2024-09-18T10:31:40.234 DEBUG:teuthology.orchestra.run.smithi134:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-09-18T10:31:40.277 INFO:teuthology.orchestra.run.smithi134.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-09-18T10:31:40.280 INFO:tasks.ceph:Setting up mds nodes... 2024-09-18T10:31:40.280 INFO:tasks.ceph_client:Setting up client nodes... 2024-09-18T10:31:40.280 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-09-18T10:31:40.311 INFO:teuthology.orchestra.run.smithi134.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-09-18T10:31:40.322 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-09-18T10:31:40.322 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi134.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}} 2024-09-18T10:31:40.323 DEBUG:teuthology.orchestra.run.smithi134:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-09-18T10:31:40.382 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-18T10:31:40.383 INFO:tasks.ceph:role: osd.0 2024-09-18T10:31:40.383 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi134.front.sepia.ceph.com 2024-09-18T10:31:40.383 DEBUG:teuthology.orchestra.run.smithi134:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-09-18T10:31:40.904 INFO:teuthology.orchestra.run.smithi134.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-09-18T10:31:40.905 INFO:teuthology.orchestra.run.smithi134.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-18T10:31:40.905 INFO:teuthology.orchestra.run.smithi134.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-18T10:31:40.905 INFO:teuthology.orchestra.run.smithi134.stdout: = reflink=1 2024-09-18T10:31:40.905 INFO:teuthology.orchestra.run.smithi134.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-18T10:31:40.905 INFO:teuthology.orchestra.run.smithi134.stdout: = sunit=0 swidth=0 blks 2024-09-18T10:31:40.906 INFO:teuthology.orchestra.run.smithi134.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-18T10:31:40.906 INFO:teuthology.orchestra.run.smithi134.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-18T10:31:40.906 INFO:teuthology.orchestra.run.smithi134.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-18T10:31:40.906 INFO:teuthology.orchestra.run.smithi134.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-18T10:31:40.908 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi134.front.sepia.ceph.com -o noatime 2024-09-18T10:31:40.908 DEBUG:teuthology.orchestra.run.smithi134:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-09-18T10:31:40.989 DEBUG:teuthology.orchestra.run.smithi134:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-09-18T10:31:41.001 INFO:teuthology.orchestra.run.smithi134.stderr:sudo: /sbin/restorecon: command not found 2024-09-18T10:31:41.002 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-18T10:31:41.002 DEBUG:teuthology.orchestra.run.smithi134:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-09-18T10:31:41.013 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-18T10:31:41.014 INFO:tasks.ceph:role: osd.1 2024-09-18T10:31:41.014 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi134.front.sepia.ceph.com 2024-09-18T10:31:41.014 DEBUG:teuthology.orchestra.run.smithi134:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-09-18T10:31:41.555 INFO:teuthology.orchestra.run.smithi134.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-09-18T10:31:41.555 INFO:teuthology.orchestra.run.smithi134.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-18T10:31:41.555 INFO:teuthology.orchestra.run.smithi134.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-18T10:31:41.556 INFO:teuthology.orchestra.run.smithi134.stdout: = reflink=1 2024-09-18T10:31:41.556 INFO:teuthology.orchestra.run.smithi134.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-18T10:31:41.556 INFO:teuthology.orchestra.run.smithi134.stdout: = sunit=0 swidth=0 blks 2024-09-18T10:31:41.556 INFO:teuthology.orchestra.run.smithi134.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-18T10:31:41.556 INFO:teuthology.orchestra.run.smithi134.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-18T10:31:41.556 INFO:teuthology.orchestra.run.smithi134.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-18T10:31:41.556 INFO:teuthology.orchestra.run.smithi134.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-18T10:31:41.557 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi134.front.sepia.ceph.com -o noatime 2024-09-18T10:31:41.557 DEBUG:teuthology.orchestra.run.smithi134:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-09-18T10:31:41.613 DEBUG:teuthology.orchestra.run.smithi134:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-09-18T10:31:41.623 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-18T10:31:41.623 INFO:teuthology.orchestra.run.smithi134.stderr:sudo: /sbin/restorecon: command not found 2024-09-18T10:31:41.623 DEBUG:teuthology.orchestra.run.smithi134:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-09-18T10:31:41.635 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-18T10:31:41.635 INFO:tasks.ceph:role: osd.2 2024-09-18T10:31:41.635 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi134.front.sepia.ceph.com 2024-09-18T10:31:41.635 DEBUG:teuthology.orchestra.run.smithi134:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-09-18T10:31:42.104 INFO:teuthology.orchestra.run.smithi134.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-09-18T10:31:42.104 INFO:teuthology.orchestra.run.smithi134.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-18T10:31:42.104 INFO:teuthology.orchestra.run.smithi134.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-18T10:31:42.104 INFO:teuthology.orchestra.run.smithi134.stdout: = reflink=1 2024-09-18T10:31:42.104 INFO:teuthology.orchestra.run.smithi134.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-18T10:31:42.104 INFO:teuthology.orchestra.run.smithi134.stdout: = sunit=0 swidth=0 blks 2024-09-18T10:31:42.104 INFO:teuthology.orchestra.run.smithi134.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-18T10:31:42.104 INFO:teuthology.orchestra.run.smithi134.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-18T10:31:42.105 INFO:teuthology.orchestra.run.smithi134.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-18T10:31:42.105 INFO:teuthology.orchestra.run.smithi134.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-18T10:31:42.106 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi134.front.sepia.ceph.com -o noatime 2024-09-18T10:31:42.106 DEBUG:teuthology.orchestra.run.smithi134:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-09-18T10:31:42.174 DEBUG:teuthology.orchestra.run.smithi134:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-09-18T10:31:42.187 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-18T10:31:42.188 INFO:teuthology.orchestra.run.smithi134.stderr:sudo: /sbin/restorecon: command not found 2024-09-18T10:31:42.189 DEBUG:teuthology.orchestra.run.smithi134:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-09-18T10:31:42.246 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-18T10:31:42.246 INFO:tasks.ceph:role: osd.3 2024-09-18T10:31:42.246 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi134.front.sepia.ceph.com 2024-09-18T10:31:42.247 DEBUG:teuthology.orchestra.run.smithi134:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-09-18T10:31:42.767 INFO:teuthology.orchestra.run.smithi134.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-09-18T10:31:42.768 INFO:teuthology.orchestra.run.smithi134.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-18T10:31:42.768 INFO:teuthology.orchestra.run.smithi134.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-18T10:31:42.768 INFO:teuthology.orchestra.run.smithi134.stdout: = reflink=1 2024-09-18T10:31:42.768 INFO:teuthology.orchestra.run.smithi134.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-18T10:31:42.768 INFO:teuthology.orchestra.run.smithi134.stdout: = sunit=0 swidth=0 blks 2024-09-18T10:31:42.768 INFO:teuthology.orchestra.run.smithi134.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-18T10:31:42.768 INFO:teuthology.orchestra.run.smithi134.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-09-18T10:31:42.768 INFO:teuthology.orchestra.run.smithi134.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-18T10:31:42.768 INFO:teuthology.orchestra.run.smithi134.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-18T10:31:42.769 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi134.front.sepia.ceph.com -o noatime 2024-09-18T10:31:42.770 DEBUG:teuthology.orchestra.run.smithi134:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2024-09-18T10:31:42.821 DEBUG:teuthology.orchestra.run.smithi134:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-09-18T10:31:42.832 INFO:teuthology.orchestra.run.smithi134.stderr:sudo: /sbin/restorecon: command not found 2024-09-18T10:31:42.833 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-18T10:31:42.833 DEBUG:teuthology.orchestra.run.smithi134:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2024-09-18T10:31:42.845 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-18T10:31:42.846 INFO:tasks.ceph:role: osd.4 2024-09-18T10:31:42.847 DEBUG:teuthology.orchestra.run.smithi134:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2024-09-18T10:31:42.857 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-18T10:31:42.857 INFO:tasks.ceph:role: osd.5 2024-09-18T10:31:42.857 DEBUG:teuthology.orchestra.run.smithi134:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2024-09-18T10:31:42.867 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-18T10:31:42.867 INFO:tasks.ceph:role: osd.6 2024-09-18T10:31:42.867 DEBUG:teuthology.orchestra.run.smithi134:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2024-09-18T10:31:42.878 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-18T10:31:42.878 INFO:tasks.ceph:role: osd.7 2024-09-18T10:31:42.878 DEBUG:teuthology.orchestra.run.smithi134:> sudo mkdir -p /var/lib/ceph/osd/ceph-8 2024-09-18T10:31:42.887 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-18T10:31:42.887 INFO:tasks.ceph:role: osd.8 2024-09-18T10:31:42.888 DEBUG:teuthology.orchestra.run.smithi134:> sudo mkdir -p /var/lib/ceph/osd/ceph-9 2024-09-18T10:31:42.945 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-18T10:31:42.945 INFO:tasks.ceph:role: osd.9 2024-09-18T10:31:42.945 DEBUG:teuthology.orchestra.run.smithi134:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-18T10:31:43.030 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-18T10:31:43.024+0000 7f22bdc03540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-09-18T10:31:43.030 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-18T10:31:43.024+0000 7f22bdc03540 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-09-18T10:31:43.031 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-18T10:31:43.028+0000 7f22bdc03540 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-09-18T10:31:43.031 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-18T10:31:43.028+0000 7f22bdc03540 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-09-18T10:31:43.031 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-18T10:31:43.028+0000 7f22bdc03540 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-09-18T10:31:43.038 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-18T10:31:43.032+0000 7f22bdc03540 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-09-18T10:31:43.598 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-18T10:31:43.592+0000 7f22bdc03540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-09-18T10:31:44.380 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-18T10:31:44.376+0000 7f22bdc03540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-09-18T10:31:45.226 DEBUG:teuthology.orchestra.run.smithi134:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-09-18T10:31:45.243 DEBUG:teuthology.orchestra.run.smithi134:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-18T10:31:45.326 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-18T10:31:45.320+0000 7f27dd217540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-09-18T10:31:45.327 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-18T10:31:45.320+0000 7f27dd217540 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-09-18T10:31:45.327 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-18T10:31:45.324+0000 7f27dd217540 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-09-18T10:31:45.327 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-18T10:31:45.324+0000 7f27dd217540 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-09-18T10:31:45.327 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-18T10:31:45.324+0000 7f27dd217540 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-09-18T10:31:45.327 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-18T10:31:45.324+0000 7f27dd217540 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-09-18T10:31:45.881 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-18T10:31:45.876+0000 7f27dd217540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-09-18T10:31:46.670 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-18T10:31:46.664+0000 7f27dd217540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-09-18T10:31:47.502 DEBUG:teuthology.orchestra.run.smithi134:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-09-18T10:31:47.519 DEBUG:teuthology.orchestra.run.smithi134:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-18T10:31:47.553 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-18T10:31:47.548+0000 7f73b6f9c540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-09-18T10:31:47.554 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-18T10:31:47.548+0000 7f73b6f9c540 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-09-18T10:31:47.554 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-18T10:31:47.548+0000 7f73b6f9c540 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-09-18T10:31:47.554 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-18T10:31:47.548+0000 7f73b6f9c540 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-09-18T10:31:47.554 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-18T10:31:47.548+0000 7f73b6f9c540 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-09-18T10:31:47.554 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-18T10:31:47.548+0000 7f73b6f9c540 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-09-18T10:31:48.118 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-18T10:31:48.112+0000 7f73b6f9c540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-09-18T10:31:48.912 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-18T10:31:48.908+0000 7f73b6f9c540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-09-18T10:31:49.745 DEBUG:teuthology.orchestra.run.smithi134:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-09-18T10:31:49.802 DEBUG:teuthology.orchestra.run.smithi134:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-18T10:31:49.838 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-18T10:31:49.832+0000 7fa19fb22540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-09-18T10:31:49.838 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-18T10:31:49.832+0000 7fa19fb22540 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-09-18T10:31:49.838 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-18T10:31:49.832+0000 7fa19fb22540 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-09-18T10:31:49.838 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-18T10:31:49.832+0000 7fa19fb22540 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-09-18T10:31:49.838 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-18T10:31:49.832+0000 7fa19fb22540 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-09-18T10:31:49.838 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-18T10:31:49.836+0000 7fa19fb22540 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-09-18T10:31:50.410 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-18T10:31:50.404+0000 7fa19fb22540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-09-18T10:31:51.197 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-18T10:31:51.192+0000 7fa19fb22540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-09-18T10:31:52.033 DEBUG:teuthology.orchestra.run.smithi134:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-09-18T10:31:52.049 DEBUG:teuthology.orchestra.run.smithi134:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-18T10:31:52.138 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-18T10:31:52.132+0000 7f9618684540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2024-09-18T10:31:52.138 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-18T10:31:52.132+0000 7f9618684540 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2024-09-18T10:31:52.138 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-18T10:31:52.132+0000 7f9618684540 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-09-18T10:31:52.138 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-18T10:31:52.132+0000 7f9618684540 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-09-18T10:31:52.138 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-18T10:31:52.132+0000 7f9618684540 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-09-18T10:31:52.138 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-18T10:31:52.132+0000 7f9618684540 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2024-09-18T10:31:52.699 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-18T10:31:52.696+0000 7f9618684540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-09-18T10:31:53.484 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-18T10:31:53.480+0000 7f9618684540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-09-18T10:31:54.329 DEBUG:teuthology.orchestra.run.smithi134:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-09-18T10:31:54.346 DEBUG:teuthology.orchestra.run.smithi134:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-18T10:31:54.383 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-18T10:31:54.380+0000 7f9cb7a3a540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2024-09-18T10:31:54.384 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-18T10:31:54.380+0000 7f9cb7a3a540 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2024-09-18T10:31:54.384 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-18T10:31:54.380+0000 7f9cb7a3a540 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-09-18T10:31:54.384 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-18T10:31:54.380+0000 7f9cb7a3a540 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-09-18T10:31:54.384 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-18T10:31:54.380+0000 7f9cb7a3a540 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-09-18T10:31:54.384 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-18T10:31:54.380+0000 7f9cb7a3a540 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2024-09-18T10:31:54.946 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-18T10:31:54.940+0000 7f9cb7a3a540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-09-18T10:31:55.727 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-18T10:31:55.724+0000 7f9cb7a3a540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-09-18T10:31:56.557 DEBUG:teuthology.orchestra.run.smithi134:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-09-18T10:31:56.617 DEBUG:teuthology.orchestra.run.smithi134:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-18T10:31:56.652 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-18T10:31:56.648+0000 7f3e59c75540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2024-09-18T10:31:56.653 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-18T10:31:56.648+0000 7f3e59c75540 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2024-09-18T10:31:56.653 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-18T10:31:56.648+0000 7f3e59c75540 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-09-18T10:31:56.653 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-18T10:31:56.648+0000 7f3e59c75540 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-09-18T10:31:56.653 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-18T10:31:56.648+0000 7f3e59c75540 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-09-18T10:31:56.653 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-18T10:31:56.648+0000 7f3e59c75540 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2024-09-18T10:31:57.208 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-18T10:31:57.204+0000 7f3e59c75540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-09-18T10:31:57.988 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-18T10:31:57.984+0000 7f3e59c75540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-09-18T10:31:58.825 DEBUG:teuthology.orchestra.run.smithi134:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2024-09-18T10:31:58.842 DEBUG:teuthology.orchestra.run.smithi134:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-18T10:31:58.876 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-18T10:31:58.872+0000 7fa6085db540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2024-09-18T10:31:58.876 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-18T10:31:58.872+0000 7fa6085db540 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2024-09-18T10:31:58.876 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-18T10:31:58.872+0000 7fa6085db540 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-09-18T10:31:58.876 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-18T10:31:58.872+0000 7fa6085db540 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-09-18T10:31:58.876 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-18T10:31:58.872+0000 7fa6085db540 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-09-18T10:31:58.876 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-18T10:31:58.872+0000 7fa6085db540 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2024-09-18T10:31:59.431 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-18T10:31:59.424+0000 7fa6085db540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-09-18T10:32:00.210 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-18T10:32:00.204+0000 7fa6085db540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-09-18T10:32:01.053 DEBUG:teuthology.orchestra.run.smithi134:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2024-09-18T10:32:01.111 DEBUG:teuthology.orchestra.run.smithi134:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 8 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-18T10:32:01.147 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-18T10:32:01.140+0000 7f5a4bcbe540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-8/keyring: can't open /var/lib/ceph/osd/ceph-8/keyring: (2) No such file or directory 2024-09-18T10:32:01.147 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-18T10:32:01.140+0000 7f5a4bcbe540 -1 created new key in keyring /var/lib/ceph/osd/ceph-8/keyring 2024-09-18T10:32:01.148 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-18T10:32:01.140+0000 7f5a4bcbe540 -1 bluestore(/var/lib/ceph/osd/ceph-8/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-8/block: (2) No such file or directory 2024-09-18T10:32:01.148 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-18T10:32:01.140+0000 7f5a4bcbe540 -1 bluestore(/var/lib/ceph/osd/ceph-8/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-8/block: (2) No such file or directory 2024-09-18T10:32:01.148 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-18T10:32:01.140+0000 7f5a4bcbe540 -1 bluestore(/var/lib/ceph/osd/ceph-8/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-8/block: (2) No such file or directory 2024-09-18T10:32:01.148 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-18T10:32:01.144+0000 7f5a4bcbe540 -1 bluestore(/var/lib/ceph/osd/ceph-8) _read_fsid unparsable uuid 2024-09-18T10:32:01.699 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-18T10:32:01.696+0000 7f5a4bcbe540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-09-18T10:32:02.496 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-18T10:32:02.492+0000 7f5a4bcbe540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-09-18T10:32:03.332 DEBUG:teuthology.orchestra.run.smithi134:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-8 2024-09-18T10:32:03.349 DEBUG:teuthology.orchestra.run.smithi134:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 9 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-18T10:32:03.433 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-18T10:32:03.428+0000 7fdf146f4540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-9/keyring: can't open /var/lib/ceph/osd/ceph-9/keyring: (2) No such file or directory 2024-09-18T10:32:03.433 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-18T10:32:03.428+0000 7fdf146f4540 -1 created new key in keyring /var/lib/ceph/osd/ceph-9/keyring 2024-09-18T10:32:03.433 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-18T10:32:03.428+0000 7fdf146f4540 -1 bluestore(/var/lib/ceph/osd/ceph-9/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-9/block: (2) No such file or directory 2024-09-18T10:32:03.433 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-18T10:32:03.428+0000 7fdf146f4540 -1 bluestore(/var/lib/ceph/osd/ceph-9/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-9/block: (2) No such file or directory 2024-09-18T10:32:03.433 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-18T10:32:03.428+0000 7fdf146f4540 -1 bluestore(/var/lib/ceph/osd/ceph-9/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-9/block: (2) No such file or directory 2024-09-18T10:32:03.434 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-18T10:32:03.428+0000 7fdf146f4540 -1 bluestore(/var/lib/ceph/osd/ceph-9) _read_fsid unparsable uuid 2024-09-18T10:32:03.995 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-18T10:32:03.992+0000 7fdf146f4540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-09-18T10:32:04.787 INFO:teuthology.orchestra.run.smithi134.stderr:2024-09-18T10:32:04.784+0000 7fdf146f4540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-09-18T10:32:05.629 DEBUG:teuthology.orchestra.run.smithi134:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-9 2024-09-18T10:32:05.646 INFO:tasks.ceph:Reading keys from all nodes... 2024-09-18T10:32:05.646 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-09-18T10:32:05.646 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-09-18T10:32:05.659 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-09-18T10:32:05.659 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-09-18T10:32:05.716 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-09-18T10:32:05.716 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-09-18T10:32:05.730 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-09-18T10:32:05.730 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-09-18T10:32:05.741 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-09-18T10:32:05.741 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-09-18T10:32:05.752 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-09-18T10:32:05.752 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2024-09-18T10:32:05.765 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-09-18T10:32:05.765 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2024-09-18T10:32:05.822 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-09-18T10:32:05.822 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2024-09-18T10:32:05.833 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-09-18T10:32:05.833 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2024-09-18T10:32:05.845 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-09-18T10:32:05.845 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd if=/var/lib/ceph/osd/ceph-8/keyring of=/dev/stdout 2024-09-18T10:32:05.906 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-09-18T10:32:05.906 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd if=/var/lib/ceph/osd/ceph-9/keyring of=/dev/stdout 2024-09-18T10:32:05.917 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-09-18T10:32:05.917 DEBUG:teuthology.orchestra.run.smithi134:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-09-18T10:32:05.924 INFO:tasks.ceph:Adding keys to all mons... 2024-09-18T10:32:05.924 DEBUG:teuthology.orchestra.run.smithi134:> sudo tee -a /etc/ceph/ceph.keyring 2024-09-18T10:32:05.980 INFO:teuthology.orchestra.run.smithi134.stdout:[mgr.x] 2024-09-18T10:32:05.980 INFO:teuthology.orchestra.run.smithi134.stdout: key = AQAMrOpmr+CWEBAAi/wYGT5BBDGzoMozPNrxng== 2024-09-18T10:32:05.981 INFO:teuthology.orchestra.run.smithi134.stdout:[osd.0] 2024-09-18T10:32:05.981 INFO:teuthology.orchestra.run.smithi134.stdout: key = AQAPrOpmHyvXARAAJ5mycgEbNTfwQ8ONl8/iKw== 2024-09-18T10:32:05.981 INFO:teuthology.orchestra.run.smithi134.stdout:[osd.1] 2024-09-18T10:32:05.981 INFO:teuthology.orchestra.run.smithi134.stdout: key = AQARrOpmk4aDExAA3ineLJzvGstkgMM6jstsbg== 2024-09-18T10:32:05.981 INFO:teuthology.orchestra.run.smithi134.stdout:[osd.2] 2024-09-18T10:32:05.981 INFO:teuthology.orchestra.run.smithi134.stdout: key = AQATrOpmAeoKIRAAn5T6jMIp2p/HndUJkCUXAg== 2024-09-18T10:32:05.981 INFO:teuthology.orchestra.run.smithi134.stdout:[osd.3] 2024-09-18T10:32:05.981 INFO:teuthology.orchestra.run.smithi134.stdout: key = AQAVrOpmpaf8MRAAMC6dbb8N73j2VzA34Ne0Pg== 2024-09-18T10:32:05.981 INFO:teuthology.orchestra.run.smithi134.stdout:[osd.4] 2024-09-18T10:32:05.981 INFO:teuthology.orchestra.run.smithi134.stdout: key = AQAYrOpmmPw6CBAAxPVs7P3J3jrYpB9G+c4Iow== 2024-09-18T10:32:05.981 INFO:teuthology.orchestra.run.smithi134.stdout:[osd.5] 2024-09-18T10:32:05.981 INFO:teuthology.orchestra.run.smithi134.stdout: key = AQAarOpm5rnoFhAASZq/+murFGg3zvOrCR9XVg== 2024-09-18T10:32:05.981 INFO:teuthology.orchestra.run.smithi134.stdout:[osd.6] 2024-09-18T10:32:05.982 INFO:teuthology.orchestra.run.smithi134.stdout: key = AQAcrOpmFHryJhAA4XaekoWcVE0O08Ct2+J+bQ== 2024-09-18T10:32:05.982 INFO:teuthology.orchestra.run.smithi134.stdout:[osd.7] 2024-09-18T10:32:05.982 INFO:teuthology.orchestra.run.smithi134.stdout: key = AQAerOpmhQpANBAAMWUyEhrUmtkkr68mGm88PQ== 2024-09-18T10:32:05.982 INFO:teuthology.orchestra.run.smithi134.stdout:[osd.8] 2024-09-18T10:32:05.982 INFO:teuthology.orchestra.run.smithi134.stdout: key = AQAhrOpmjp/DCBAAzaqtcnqrvHWXsShmBCg6Fg== 2024-09-18T10:32:05.982 INFO:teuthology.orchestra.run.smithi134.stdout:[osd.9] 2024-09-18T10:32:05.982 INFO:teuthology.orchestra.run.smithi134.stdout: key = AQAjrOpmI5bYGRAAtpX2VYTnId89/IN4E3f/Xg== 2024-09-18T10:32:05.982 INFO:teuthology.orchestra.run.smithi134.stdout:[client.0] 2024-09-18T10:32:05.982 INFO:teuthology.orchestra.run.smithi134.stdout: key = AQAMrOpms7qXEhAAXcFxrjhld3eHQPoIPCbPQw== 2024-09-18T10:32:05.983 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-18T10:32:06.024 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-18T10:32:06.064 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-18T10:32:06.150 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-18T10:32:06.192 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-18T10:32:06.232 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-18T10:32:06.274 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-18T10:32:06.358 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-18T10:32:06.401 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-18T10:32:06.442 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.8 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-18T10:32:06.483 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.9 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-18T10:32:06.573 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-18T10:32:06.617 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-09-18T10:32:06.617 DEBUG:teuthology.orchestra.run.smithi134:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-09-18T10:32:06.630 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-18T10:32:06.681 DEBUG:teuthology.orchestra.run.smithi134:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-09-18T10:32:06.693 DEBUG:teuthology.orchestra.run.smithi134:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-09-18T10:32:06.697 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-09-18T10:32:06.697 INFO:tasks.ceph.mon.a:Restarting daemon 2024-09-18T10:32:06.698 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-09-18T10:32:06.700 INFO:tasks.ceph.mon.a:Started 2024-09-18T10:32:06.700 DEBUG:teuthology.orchestra.run.smithi134:> sudo ceph config set mgr mgr_pool false --force 2024-09-18T10:32:06.928 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-09-18T10:32:06.928 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-09-18T10:32:06.928 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-09-18T10:32:06.931 INFO:tasks.ceph.mgr.x:Started 2024-09-18T10:32:06.931 INFO:tasks.ceph:Setting crush tunables to default 2024-09-18T10:32:06.931 DEBUG:teuthology.orchestra.run.smithi134:> sudo ceph --cluster ceph osd crush tunables default 2024-09-18T10:32:07.126 INFO:teuthology.orchestra.run.smithi134.stderr:adjusted tunables profile to default 2024-09-18T10:32:07.136 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-09-18T10:32:07.136 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-09-18T10:32:07.136 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-09-18T10:32:07.145 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-09-18T10:32:07.145 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-09-18T10:32:07.156 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-09-18T10:32:07.156 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-09-18T10:32:07.213 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-09-18T10:32:07.213 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-09-18T10:32:07.222 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-09-18T10:32:07.222 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2024-09-18T10:32:07.225 INFO:tasks.ceph.mgr.x.smithi134.stderr:2024-09-18T10:32:07.220+0000 7fef31127000 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-18T10:32:07.232 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-09-18T10:32:07.232 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2024-09-18T10:32:07.293 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-09-18T10:32:07.293 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2024-09-18T10:32:07.302 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-09-18T10:32:07.302 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2024-09-18T10:32:07.312 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-09-18T10:32:07.312 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd if=/var/lib/ceph/osd/ceph-8/fsid of=/dev/stdout 2024-09-18T10:32:07.367 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-09-18T10:32:07.367 DEBUG:teuthology.orchestra.run.smithi134:> sudo dd if=/var/lib/ceph/osd/ceph-9/fsid of=/dev/stdout 2024-09-18T10:32:07.381 DEBUG:teuthology.orchestra.run.smithi134:> sudo ceph --cluster ceph osd new 5d600f22-6bed-4106-a48d-b8af006ace47 0 2024-09-18T10:32:07.595 INFO:teuthology.orchestra.run.smithi134.stdout:0 2024-09-18T10:32:07.608 DEBUG:teuthology.orchestra.run.smithi134:> sudo ceph --cluster ceph osd new 17de89fa-8f8b-495e-8fd6-24d62bfbb4de 1 2024-09-18T10:32:07.805 INFO:teuthology.orchestra.run.smithi134.stdout:1 2024-09-18T10:32:07.817 DEBUG:teuthology.orchestra.run.smithi134:> sudo ceph --cluster ceph osd new 19059e59-5e31-4f29-83a3-9b068d17b228 2 2024-09-18T10:32:07.950 INFO:tasks.ceph.mgr.x.smithi134.stderr:2024-09-18T10:32:07.944+0000 7fef31127000 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-18T10:32:08.006 INFO:tasks.ceph.mgr.x.smithi134.stderr:2024-09-18T10:32:08.000+0000 7fef31127000 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-18T10:32:08.011 INFO:teuthology.orchestra.run.smithi134.stdout:2 2024-09-18T10:32:08.024 DEBUG:teuthology.orchestra.run.smithi134:> sudo ceph --cluster ceph osd new 636276c9-3bda-4382-95c1-03bb98e80ac3 3 2024-09-18T10:32:08.124 INFO:tasks.ceph.mgr.x.smithi134.stderr:2024-09-18T10:32:08.120+0000 7fef31127000 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-18T10:32:08.221 INFO:teuthology.orchestra.run.smithi134.stdout:3 2024-09-18T10:32:08.237 DEBUG:teuthology.orchestra.run.smithi134:> sudo ceph --cluster ceph osd new b918785a-5612-49ab-9b71-78433ef817f0 4 2024-09-18T10:32:08.351 INFO:tasks.ceph.mgr.x.smithi134.stderr:2024-09-18T10:32:08.348+0000 7fef31127000 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-18T10:32:08.401 INFO:tasks.ceph.mgr.x.smithi134.stderr:2024-09-18T10:32:08.396+0000 7fef31127000 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-18T10:32:08.430 INFO:teuthology.orchestra.run.smithi134.stdout:4 2024-09-18T10:32:08.442 DEBUG:teuthology.orchestra.run.smithi134:> sudo ceph --cluster ceph osd new 4b051f94-3f16-4114-83dc-2ea21f6da775 5 2024-09-18T10:32:08.581 INFO:tasks.ceph.mgr.x.smithi134.stderr:2024-09-18T10:32:08.576+0000 7fef31127000 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-18T10:32:08.633 INFO:teuthology.orchestra.run.smithi134.stdout:5 2024-09-18T10:32:08.641 INFO:tasks.ceph.mgr.x.smithi134.stderr:2024-09-18T10:32:08.636+0000 7fef31127000 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-18T10:32:08.644 DEBUG:teuthology.orchestra.run.smithi134:> sudo ceph --cluster ceph osd new 7771b052-b691-4ede-9703-fa5a4bc2c6e2 6 2024-09-18T10:32:08.704 INFO:tasks.ceph.mgr.x.smithi134.stderr:2024-09-18T10:32:08.700+0000 7fef31127000 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-18T10:32:08.833 INFO:teuthology.orchestra.run.smithi134.stdout:6 2024-09-18T10:32:08.844 DEBUG:teuthology.orchestra.run.smithi134:> sudo ceph --cluster ceph osd new e956d2c2-b6e6-422f-a052-5e7efe34ce26 7 2024-09-18T10:32:09.031 INFO:teuthology.orchestra.run.smithi134.stdout:7 2024-09-18T10:32:09.042 DEBUG:teuthology.orchestra.run.smithi134:> sudo ceph --cluster ceph osd new 1896fcc5-d899-4784-8a2c-11344b1860e7 8 2024-09-18T10:32:09.103 INFO:tasks.ceph.mgr.x.smithi134.stderr:2024-09-18T10:32:09.100+0000 7fef31127000 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-18T10:32:09.215 INFO:tasks.ceph.mgr.x.smithi134.stderr:2024-09-18T10:32:09.212+0000 7fef31127000 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-18T10:32:09.225 INFO:teuthology.orchestra.run.smithi134.stdout:8 2024-09-18T10:32:09.238 DEBUG:teuthology.orchestra.run.smithi134:> sudo ceph --cluster ceph osd new 818002b3-6bbd-4802-95e5-eaef0a009ccb 9 2024-09-18T10:32:09.432 INFO:teuthology.orchestra.run.smithi134.stdout:9 2024-09-18T10:32:09.444 INFO:tasks.ceph.osd.0:Restarting daemon 2024-09-18T10:32:09.445 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-09-18T10:32:09.446 INFO:tasks.ceph.osd.0:Started 2024-09-18T10:32:09.447 INFO:tasks.ceph.osd.1:Restarting daemon 2024-09-18T10:32:09.447 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-09-18T10:32:09.448 INFO:tasks.ceph.osd.1:Started 2024-09-18T10:32:09.448 INFO:tasks.ceph.osd.2:Restarting daemon 2024-09-18T10:32:09.448 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-09-18T10:32:09.450 INFO:tasks.ceph.osd.2:Started 2024-09-18T10:32:09.451 INFO:tasks.ceph.osd.3:Restarting daemon 2024-09-18T10:32:09.451 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-09-18T10:32:09.452 INFO:tasks.ceph.osd.3:Started 2024-09-18T10:32:09.452 INFO:tasks.ceph.osd.4:Restarting daemon 2024-09-18T10:32:09.453 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2024-09-18T10:32:09.455 INFO:tasks.ceph.osd.4:Started 2024-09-18T10:32:09.455 INFO:tasks.ceph.osd.5:Restarting daemon 2024-09-18T10:32:09.455 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-09-18T10:32:09.457 INFO:tasks.ceph.osd.5:Started 2024-09-18T10:32:09.458 INFO:tasks.ceph.osd.6:Restarting daemon 2024-09-18T10:32:09.458 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2024-09-18T10:32:09.459 INFO:tasks.ceph.osd.6:Started 2024-09-18T10:32:09.460 INFO:tasks.ceph.osd.7:Restarting daemon 2024-09-18T10:32:09.460 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2024-09-18T10:32:09.462 INFO:tasks.ceph.osd.7:Started 2024-09-18T10:32:09.462 INFO:tasks.ceph.osd.8:Restarting daemon 2024-09-18T10:32:09.462 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 8 2024-09-18T10:32:09.464 INFO:tasks.ceph.osd.8:Started 2024-09-18T10:32:09.464 INFO:tasks.ceph.osd.9:Restarting daemon 2024-09-18T10:32:09.464 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 9 2024-09-18T10:32:09.466 INFO:tasks.ceph.osd.9:Started 2024-09-18T10:32:09.466 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-18T10:32:09.566 INFO:tasks.ceph.mgr.x.smithi134.stderr:2024-09-18T10:32:09.560+0000 7fef31127000 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-18T10:32:09.707 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-18T10:32:09.707 INFO:teuthology.orchestra.run.smithi134.stdout:{"epoch":12,"fsid":"909a0a21-79ad-49e6-bd09-2f76d1ea12ab","created":"2024-09-18T10:32:06.815161+0000","modified":"2024-09-18T10:32:09.430449+0000","last_up_change":"0.000000","last_in_change":"2024-09-18T10:32:09.430449+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":10,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[],"osds":[{"osd":0,"uuid":"5d600f22-6bed-4106-a48d-b8af006ace47","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"17de89fa-8f8b-495e-8fd6-24d62bfbb4de","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"19059e59-5e31-4f29-83a3-9b068d17b228","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"636276c9-3bda-4382-95c1-03bb98e80ac3","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"b918785a-5612-49ab-9b71-78433ef817f0","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"4b051f94-3f16-4114-83dc-2ea21f6da775","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"7771b052-b691-4ede-9703-fa5a4bc2c6e2","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"e956d2c2-b6e6-422f-a052-5e7efe34ce26","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":8,"uuid":"1896fcc5-d899-4784-8a2c-11344b1860e7","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":9,"uuid":"818002b3-6bbd-4802-95e5-eaef0a009ccb","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-18T10:32:09.718 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-09-18T10:32:09.718 INFO:tasks.ceph:Waiting for OSDs to come up 2024-09-18T10:32:09.760 INFO:tasks.ceph.mgr.x.smithi134.stderr:2024-09-18T10:32:09.752+0000 7fef31127000 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-18T10:32:09.830 INFO:tasks.ceph.mgr.x.smithi134.stderr:2024-09-18T10:32:09.824+0000 7fef31127000 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-18T10:32:09.856 INFO:tasks.ceph.osd.2.smithi134.stderr:2024-09-18T10:32:09.852+0000 7fcdd1751540 -1 Falling back to public interface 2024-09-18T10:32:09.864 INFO:tasks.ceph.osd.4.smithi134.stderr:2024-09-18T10:32:09.860+0000 7ff00e294540 -1 Falling back to public interface 2024-09-18T10:32:09.865 INFO:tasks.ceph.osd.1.smithi134.stderr:2024-09-18T10:32:09.860+0000 7ff3293c6540 -1 Falling back to public interface 2024-09-18T10:32:09.865 INFO:tasks.ceph.osd.3.smithi134.stderr:2024-09-18T10:32:09.860+0000 7fc9dfe47540 -1 Falling back to public interface 2024-09-18T10:32:09.875 INFO:tasks.ceph.osd.5.smithi134.stderr:2024-09-18T10:32:09.872+0000 7fc7c685e540 -1 Falling back to public interface 2024-09-18T10:32:09.876 INFO:tasks.ceph.osd.0.smithi134.stderr:2024-09-18T10:32:09.872+0000 7fd46cb59540 -1 Falling back to public interface 2024-09-18T10:32:09.876 INFO:tasks.ceph.osd.9.smithi134.stderr:2024-09-18T10:32:09.872+0000 7f20dc08d540 -1 Falling back to public interface 2024-09-18T10:32:09.891 INFO:tasks.ceph.mgr.x.smithi134.stderr:2024-09-18T10:32:09.888+0000 7fef31127000 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-18T10:32:09.912 INFO:tasks.ceph.osd.6.smithi134.stderr:2024-09-18T10:32:09.908+0000 7fa9afd62540 -1 Falling back to public interface 2024-09-18T10:32:09.951 INFO:tasks.ceph.mgr.x.smithi134.stderr:2024-09-18T10:32:09.948+0000 7fef31127000 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-18T10:32:10.121 INFO:tasks.ceph.osd.8.smithi134.stderr:2024-09-18T10:32:10.116+0000 7fd07e1ad540 -1 Falling back to public interface 2024-09-18T10:32:10.121 INFO:tasks.ceph.osd.7.smithi134.stderr:2024-09-18T10:32:10.116+0000 7f2de7499540 -1 Falling back to public interface 2024-09-18T10:32:10.134 INFO:tasks.ceph.mgr.x.smithi134.stderr:2024-09-18T10:32:10.132+0000 7fef31127000 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-18T10:32:10.315 INFO:tasks.ceph.mgr.x.smithi134.stderr:2024-09-18T10:32:10.312+0000 7fef31127000 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-18T10:32:10.387 INFO:tasks.ceph.mgr.x.smithi134.stderr:2024-09-18T10:32:10.384+0000 7fef31127000 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-18T10:32:10.562 INFO:tasks.ceph.mgr.x.smithi134.stderr:2024-09-18T10:32:10.556+0000 7fef31127000 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-18T10:32:10.636 INFO:tasks.ceph.mgr.x.smithi134.stderr:2024-09-18T10:32:10.632+0000 7fef31127000 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-18T10:32:10.724 DEBUG:teuthology.orchestra.run.smithi134:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-18T10:32:10.892 INFO:teuthology.misc.health.smithi134.stdout: 2024-09-18T10:32:10.892 INFO:teuthology.misc.health.smithi134.stdout:{"epoch":12,"fsid":"909a0a21-79ad-49e6-bd09-2f76d1ea12ab","created":"2024-09-18T10:32:06.815161+0000","modified":"2024-09-18T10:32:09.430449+0000","last_up_change":"0.000000","last_in_change":"2024-09-18T10:32:09.430449+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":10,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[],"osds":[{"osd":0,"uuid":"5d600f22-6bed-4106-a48d-b8af006ace47","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"17de89fa-8f8b-495e-8fd6-24d62bfbb4de","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"19059e59-5e31-4f29-83a3-9b068d17b228","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"636276c9-3bda-4382-95c1-03bb98e80ac3","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"b918785a-5612-49ab-9b71-78433ef817f0","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"4b051f94-3f16-4114-83dc-2ea21f6da775","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"7771b052-b691-4ede-9703-fa5a4bc2c6e2","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"e956d2c2-b6e6-422f-a052-5e7efe34ce26","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":8,"uuid":"1896fcc5-d899-4784-8a2c-11344b1860e7","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":9,"uuid":"818002b3-6bbd-4802-95e5-eaef0a009ccb","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-18T10:32:10.902 DEBUG:teuthology.misc:0 of 10 OSDs are up 2024-09-18T10:32:11.374 INFO:tasks.ceph.mgr.x.smithi134.stderr:2024-09-18T10:32:11.368+0000 7fef31127000 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-18T10:32:12.408 INFO:tasks.ceph.mgr.x.smithi134.stderr:2024-09-18T10:32:12.404+0000 7fef1a19e700 -1 mgr.server handle_report got status from non-daemon mon.a 2024-09-18T10:32:12.416 INFO:tasks.ceph.osd.3.smithi134.stderr:2024-09-18T10:32:12.412+0000 7fc9dfe47540 -1 osd.3 0 log_to_monitors true 2024-09-18T10:32:12.438 INFO:tasks.ceph.osd.9.smithi134.stderr:2024-09-18T10:32:12.432+0000 7f20dc08d540 -1 osd.9 0 log_to_monitors true 2024-09-18T10:32:12.624 INFO:tasks.ceph.osd.0.smithi134.stderr:2024-09-18T10:32:12.620+0000 7fd46cb59540 -1 osd.0 0 log_to_monitors true 2024-09-18T10:32:12.661 INFO:tasks.ceph.osd.5.smithi134.stderr:2024-09-18T10:32:12.656+0000 7fc7c685e540 -1 osd.5 0 log_to_monitors true 2024-09-18T10:32:12.662 INFO:tasks.ceph.osd.1.smithi134.stderr:2024-09-18T10:32:12.656+0000 7ff3293c6540 -1 osd.1 0 log_to_monitors true 2024-09-18T10:32:12.688 INFO:tasks.ceph.osd.2.smithi134.stderr:2024-09-18T10:32:12.684+0000 7fcdd1751540 -1 osd.2 0 log_to_monitors true 2024-09-18T10:32:12.719 INFO:tasks.ceph.osd.6.smithi134.stderr:2024-09-18T10:32:12.716+0000 7fa9afd62540 -1 osd.6 0 log_to_monitors true 2024-09-18T10:32:12.889 INFO:tasks.ceph.osd.4.smithi134.stderr:2024-09-18T10:32:12.884+0000 7ff00e294540 -1 osd.4 0 log_to_monitors true 2024-09-18T10:32:13.126 INFO:tasks.ceph.osd.7.smithi134.stderr:2024-09-18T10:32:13.120+0000 7f2de7499540 -1 osd.7 0 log_to_monitors true 2024-09-18T10:32:13.131 INFO:tasks.ceph.osd.8.smithi134.stderr:2024-09-18T10:32:13.128+0000 7fd07e1ad540 -1 osd.8 0 log_to_monitors true 2024-09-18T10:32:14.426 INFO:tasks.ceph.osd.5.smithi134.stderr:2024-09-18T10:32:14.419+0000 7fc7c1ffe700 -1 osd.5 0 waiting for initial osdmap 2024-09-18T10:32:14.427 INFO:tasks.ceph.osd.2.smithi134.stderr:2024-09-18T10:32:14.423+0000 7fcdccef1700 -1 osd.2 0 waiting for initial osdmap 2024-09-18T10:32:14.428 INFO:tasks.ceph.osd.9.smithi134.stderr:2024-09-18T10:32:14.423+0000 7f20d782d700 -1 osd.9 0 waiting for initial osdmap 2024-09-18T10:32:14.428 INFO:tasks.ceph.osd.1.smithi134.stderr:2024-09-18T10:32:14.423+0000 7ff324b66700 -1 osd.1 0 waiting for initial osdmap 2024-09-18T10:32:14.428 INFO:tasks.ceph.osd.7.smithi134.stderr:2024-09-18T10:32:14.423+0000 7f2de2c39700 -1 osd.7 0 waiting for initial osdmap 2024-09-18T10:32:14.428 INFO:tasks.ceph.osd.3.smithi134.stderr:2024-09-18T10:32:14.419+0000 7fc9db5e7700 -1 osd.3 0 waiting for initial osdmap 2024-09-18T10:32:14.429 INFO:tasks.ceph.osd.8.smithi134.stderr:2024-09-18T10:32:14.423+0000 7fd07994d700 -1 osd.8 0 waiting for initial osdmap 2024-09-18T10:32:14.429 INFO:tasks.ceph.osd.4.smithi134.stderr:2024-09-18T10:32:14.423+0000 7ff00b259700 -1 osd.4 0 waiting for initial osdmap 2024-09-18T10:32:14.430 INFO:tasks.ceph.osd.0.smithi134.stderr:2024-09-18T10:32:14.423+0000 7fd469b1e700 -1 osd.0 0 waiting for initial osdmap 2024-09-18T10:32:14.430 INFO:tasks.ceph.osd.6.smithi134.stderr:2024-09-18T10:32:14.423+0000 7fa9ab502700 -1 osd.6 0 waiting for initial osdmap 2024-09-18T10:32:14.436 INFO:tasks.ceph.osd.3.smithi134.stderr:2024-09-18T10:32:14.431+0000 7fc9d7b91700 -1 osd.3 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-18T10:32:14.437 INFO:tasks.ceph.osd.1.smithi134.stderr:2024-09-18T10:32:14.431+0000 7ff32090f700 -1 osd.1 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-18T10:32:14.438 INFO:tasks.ceph.osd.2.smithi134.stderr:2024-09-18T10:32:14.431+0000 7fcdc8c9a700 -1 osd.2 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-18T10:32:14.440 INFO:tasks.ceph.osd.7.smithi134.stderr:2024-09-18T10:32:14.435+0000 7f2dde1e1700 -1 osd.7 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-18T10:32:14.440 INFO:tasks.ceph.osd.5.smithi134.stderr:2024-09-18T10:32:14.435+0000 7fc7bdda7700 -1 osd.5 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-18T10:32:14.441 INFO:tasks.ceph.osd.4.smithi134.stderr:2024-09-18T10:32:14.435+0000 7ff0057dd700 -1 osd.4 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-18T10:32:14.442 INFO:tasks.ceph.osd.6.smithi134.stderr:2024-09-18T10:32:14.435+0000 7fa9a72ab700 -1 osd.6 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-18T10:32:14.442 INFO:tasks.ceph.osd.0.smithi134.stderr:2024-09-18T10:32:14.435+0000 7fd4648a3700 -1 osd.0 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-18T10:32:14.442 INFO:tasks.ceph.osd.9.smithi134.stderr:2024-09-18T10:32:14.435+0000 7f20d3dd7700 -1 osd.9 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-18T10:32:14.443 INFO:tasks.ceph.osd.8.smithi134.stderr:2024-09-18T10:32:14.435+0000 7fd074ef5700 -1 osd.8 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-18T10:32:17.908 DEBUG:teuthology.orchestra.run.smithi134:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-18T10:32:18.198 INFO:teuthology.misc.health.smithi134.stdout: 2024-09-18T10:32:18.198 INFO:teuthology.misc.health.smithi134.stdout:{"epoch":16,"fsid":"909a0a21-79ad-49e6-bd09-2f76d1ea12ab","created":"2024-09-18T10:32:06.815161+0000","modified":"2024-09-18T10:32:16.432210+0000","last_up_change":"2024-09-18T10:32:15.421636+0000","last_in_change":"2024-09-18T10:32:09.430449+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":10,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[],"osds":[{"osd":0,"uuid":"5d600f22-6bed-4106-a48d-b8af006ace47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":25641},{"type":"v1","addr":"172.21.15.134:6837","nonce":25641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6840","nonce":25641},{"type":"v1","addr":"172.21.15.134:6843","nonce":25641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6853","nonce":25641},{"type":"v1","addr":"172.21.15.134:6855","nonce":25641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6847","nonce":25641},{"type":"v1","addr":"172.21.15.134:6850","nonce":25641}]},"public_addr":"172.21.15.134:6837/25641","cluster_addr":"172.21.15.134:6843/25641","heartbeat_back_addr":"172.21.15.134:6855/25641","heartbeat_front_addr":"172.21.15.134:6850/25641","state":["exists","up"]},{"osd":1,"uuid":"17de89fa-8f8b-495e-8fd6-24d62bfbb4de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":25636},{"type":"v1","addr":"172.21.15.134:6813","nonce":25636}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6817","nonce":25636},{"type":"v1","addr":"172.21.15.134:6820","nonce":25636}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6827","nonce":25636},{"type":"v1","addr":"172.21.15.134:6830","nonce":25636}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":25636},{"type":"v1","addr":"172.21.15.134:6825","nonce":25636}]},"public_addr":"172.21.15.134:6813/25636","cluster_addr":"172.21.15.134:6820/25636","heartbeat_back_addr":"172.21.15.134:6830/25636","heartbeat_front_addr":"172.21.15.134:6825/25636","state":["exists","up"]},{"osd":2,"uuid":"19059e59-5e31-4f29-83a3-9b068d17b228","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":25637},{"type":"v1","addr":"172.21.15.134:6801","nonce":25637}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":25637},{"type":"v1","addr":"172.21.15.134:6803","nonce":25637}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":25637},{"type":"v1","addr":"172.21.15.134:6807","nonce":25637}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":25637},{"type":"v1","addr":"172.21.15.134:6805","nonce":25637}]},"public_addr":"172.21.15.134:6801/25637","cluster_addr":"172.21.15.134:6803/25637","heartbeat_back_addr":"172.21.15.134:6807/25637","heartbeat_front_addr":"172.21.15.134:6805/25637","state":["exists","up"]},{"osd":3,"uuid":"636276c9-3bda-4382-95c1-03bb98e80ac3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":25639},{"type":"v1","addr":"172.21.15.134:6814","nonce":25639}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":25639},{"type":"v1","addr":"172.21.15.134:6818","nonce":25639}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":25639},{"type":"v1","addr":"172.21.15.134:6829","nonce":25639}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6821","nonce":25639},{"type":"v1","addr":"172.21.15.134:6824","nonce":25639}]},"public_addr":"172.21.15.134:6814/25639","cluster_addr":"172.21.15.134:6818/25639","heartbeat_back_addr":"172.21.15.134:6829/25639","heartbeat_front_addr":"172.21.15.134:6824/25639","state":["exists","up"]},{"osd":4,"uuid":"b918785a-5612-49ab-9b71-78433ef817f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":25640},{"type":"v1","addr":"172.21.15.134:6809","nonce":25640}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6811","nonce":25640},{"type":"v1","addr":"172.21.15.134:6815","nonce":25640}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":25640},{"type":"v1","addr":"172.21.15.134:6831","nonce":25640}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6819","nonce":25640},{"type":"v1","addr":"172.21.15.134:6823","nonce":25640}]},"public_addr":"172.21.15.134:6809/25640","cluster_addr":"172.21.15.134:6815/25640","heartbeat_back_addr":"172.21.15.134:6831/25640","heartbeat_front_addr":"172.21.15.134:6823/25640","state":["exists","up"]},{"osd":5,"uuid":"4b051f94-3f16-4114-83dc-2ea21f6da775","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":25645},{"type":"v1","addr":"172.21.15.134:6835","nonce":25645}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":25645},{"type":"v1","addr":"172.21.15.134:6841","nonce":25645}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6849","nonce":25645},{"type":"v1","addr":"172.21.15.134:6851","nonce":25645}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6844","nonce":25645},{"type":"v1","addr":"172.21.15.134:6846","nonce":25645}]},"public_addr":"172.21.15.134:6835/25645","cluster_addr":"172.21.15.134:6841/25645","heartbeat_back_addr":"172.21.15.134:6851/25645","heartbeat_front_addr":"172.21.15.134:6846/25645","state":["exists","up"]},{"osd":6,"uuid":"7771b052-b691-4ede-9703-fa5a4bc2c6e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6856","nonce":25767},{"type":"v1","addr":"172.21.15.134:6857","nonce":25767}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6858","nonce":25767},{"type":"v1","addr":"172.21.15.134:6859","nonce":25767}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6862","nonce":25767},{"type":"v1","addr":"172.21.15.134:6863","nonce":25767}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6860","nonce":25767},{"type":"v1","addr":"172.21.15.134:6861","nonce":25767}]},"public_addr":"172.21.15.134:6857/25767","cluster_addr":"172.21.15.134:6859/25767","heartbeat_back_addr":"172.21.15.134:6863/25767","heartbeat_front_addr":"172.21.15.134:6861/25767","state":["exists","up"]},{"osd":7,"uuid":"e956d2c2-b6e6-422f-a052-5e7efe34ce26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6864","nonce":25642},{"type":"v1","addr":"172.21.15.134:6866","nonce":25642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6868","nonce":25642},{"type":"v1","addr":"172.21.15.134:6870","nonce":25642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6876","nonce":25642},{"type":"v1","addr":"172.21.15.134:6878","nonce":25642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6872","nonce":25642},{"type":"v1","addr":"172.21.15.134:6874","nonce":25642}]},"public_addr":"172.21.15.134:6866/25642","cluster_addr":"172.21.15.134:6870/25642","heartbeat_back_addr":"172.21.15.134:6878/25642","heartbeat_front_addr":"172.21.15.134:6874/25642","state":["exists","up"]},{"osd":8,"uuid":"1896fcc5-d899-4784-8a2c-11344b1860e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6865","nonce":25694},{"type":"v1","addr":"172.21.15.134:6867","nonce":25694}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6869","nonce":25694},{"type":"v1","addr":"172.21.15.134:6871","nonce":25694}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6877","nonce":25694},{"type":"v1","addr":"172.21.15.134:6879","nonce":25694}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6873","nonce":25694},{"type":"v1","addr":"172.21.15.134:6875","nonce":25694}]},"public_addr":"172.21.15.134:6867/25694","cluster_addr":"172.21.15.134:6871/25694","heartbeat_back_addr":"172.21.15.134:6879/25694","heartbeat_front_addr":"172.21.15.134:6875/25694","state":["exists","up"]},{"osd":9,"uuid":"818002b3-6bbd-4802-95e5-eaef0a009ccb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6833","nonce":25649},{"type":"v1","addr":"172.21.15.134:6836","nonce":25649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6839","nonce":25649},{"type":"v1","addr":"172.21.15.134:6842","nonce":25649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6852","nonce":25649},{"type":"v1","addr":"172.21.15.134:6854","nonce":25649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6845","nonce":25649},{"type":"v1","addr":"172.21.15.134:6848","nonce":25649}]},"public_addr":"172.21.15.134:6836/25649","cluster_addr":"172.21.15.134:6842/25649","heartbeat_back_addr":"172.21.15.134:6854/25649","heartbeat_front_addr":"172.21.15.134:6848/25649","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T10:32:13.629735+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T10:32:13.634602+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T10:32:13.644041+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T10:32:13.431697+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T10:32:13.848559+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T10:32:13.632248+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T10:32:13.676354+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T10:32:14.111210+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T10:32:14.173467+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T10:32:13.436859+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-18T10:32:18.209 DEBUG:teuthology.misc:10 of 10 OSDs are up 2024-09-18T10:32:18.210 INFO:tasks.ceph:Creating RBD pool 2024-09-18T10:32:18.210 DEBUG:teuthology.orchestra.run.smithi134:> sudo ceph --cluster ceph osd pool create rbd 8 2024-09-18T10:32:19.458 INFO:teuthology.orchestra.run.smithi134.stderr:pool 'rbd' created 2024-09-18T10:32:19.471 DEBUG:teuthology.orchestra.run.smithi134:> rbd --cluster ceph pool init rbd 2024-09-18T10:32:22.493 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-09-18T10:32:22.494 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-09-18T10:32:22.494 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-09-18T10:32:22.494 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-09-18T10:32:22.494 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-09-18T10:32:22.836 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-18T10:32:22.849 INFO:teuthology.orchestra.run.smithi134.stdout:{"epoch":4,"flags":0,"active_gid":4101,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6880","nonce":25269},{"type":"v1","addr":"172.21.15.134:6881","nonce":25269}]},"active_addr":"172.21.15.134:6881/25269","active_change":"2024-09-18T10:32:11.393456+0000","active_mgr_features":4540138320759226367,"available":true,"standbys":[],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":false,"error_string":"loading asyncssh library:No module named 'asyncssh'","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.1.5","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.134:0","nonce":1755667034}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.134:0","nonce":2644847161}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.134:0","nonce":4136415850}]}]} 2024-09-18T10:32:22.853 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-09-18T10:32:22.853 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-09-18T10:32:22.853 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-18T10:32:23.140 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-18T10:32:23.141 INFO:teuthology.orchestra.run.smithi134.stdout:{"epoch":20,"fsid":"909a0a21-79ad-49e6-bd09-2f76d1ea12ab","created":"2024-09-18T10:32:06.815161+0000","modified":"2024-09-18T10:32:22.466240+0000","last_up_change":"2024-09-18T10:32:15.421636+0000","last_in_change":"2024-09-18T10:32:09.430449+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":10,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2024-09-18T10:32:18.531989+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":20,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"5d600f22-6bed-4106-a48d-b8af006ace47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":25641},{"type":"v1","addr":"172.21.15.134:6837","nonce":25641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6840","nonce":25641},{"type":"v1","addr":"172.21.15.134:6843","nonce":25641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6853","nonce":25641},{"type":"v1","addr":"172.21.15.134:6855","nonce":25641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6847","nonce":25641},{"type":"v1","addr":"172.21.15.134:6850","nonce":25641}]},"public_addr":"172.21.15.134:6837/25641","cluster_addr":"172.21.15.134:6843/25641","heartbeat_back_addr":"172.21.15.134:6855/25641","heartbeat_front_addr":"172.21.15.134:6850/25641","state":["exists","up"]},{"osd":1,"uuid":"17de89fa-8f8b-495e-8fd6-24d62bfbb4de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":25636},{"type":"v1","addr":"172.21.15.134:6813","nonce":25636}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6817","nonce":25636},{"type":"v1","addr":"172.21.15.134:6820","nonce":25636}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6827","nonce":25636},{"type":"v1","addr":"172.21.15.134:6830","nonce":25636}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":25636},{"type":"v1","addr":"172.21.15.134:6825","nonce":25636}]},"public_addr":"172.21.15.134:6813/25636","cluster_addr":"172.21.15.134:6820/25636","heartbeat_back_addr":"172.21.15.134:6830/25636","heartbeat_front_addr":"172.21.15.134:6825/25636","state":["exists","up"]},{"osd":2,"uuid":"19059e59-5e31-4f29-83a3-9b068d17b228","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":25637},{"type":"v1","addr":"172.21.15.134:6801","nonce":25637}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":25637},{"type":"v1","addr":"172.21.15.134:6803","nonce":25637}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":25637},{"type":"v1","addr":"172.21.15.134:6807","nonce":25637}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":25637},{"type":"v1","addr":"172.21.15.134:6805","nonce":25637}]},"public_addr":"172.21.15.134:6801/25637","cluster_addr":"172.21.15.134:6803/25637","heartbeat_back_addr":"172.21.15.134:6807/25637","heartbeat_front_addr":"172.21.15.134:6805/25637","state":["exists","up"]},{"osd":3,"uuid":"636276c9-3bda-4382-95c1-03bb98e80ac3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":25639},{"type":"v1","addr":"172.21.15.134:6814","nonce":25639}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":25639},{"type":"v1","addr":"172.21.15.134:6818","nonce":25639}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":25639},{"type":"v1","addr":"172.21.15.134:6829","nonce":25639}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6821","nonce":25639},{"type":"v1","addr":"172.21.15.134:6824","nonce":25639}]},"public_addr":"172.21.15.134:6814/25639","cluster_addr":"172.21.15.134:6818/25639","heartbeat_back_addr":"172.21.15.134:6829/25639","heartbeat_front_addr":"172.21.15.134:6824/25639","state":["exists","up"]},{"osd":4,"uuid":"b918785a-5612-49ab-9b71-78433ef817f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":25640},{"type":"v1","addr":"172.21.15.134:6809","nonce":25640}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6811","nonce":25640},{"type":"v1","addr":"172.21.15.134:6815","nonce":25640}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":25640},{"type":"v1","addr":"172.21.15.134:6831","nonce":25640}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6819","nonce":25640},{"type":"v1","addr":"172.21.15.134:6823","nonce":25640}]},"public_addr":"172.21.15.134:6809/25640","cluster_addr":"172.21.15.134:6815/25640","heartbeat_back_addr":"172.21.15.134:6831/25640","heartbeat_front_addr":"172.21.15.134:6823/25640","state":["exists","up"]},{"osd":5,"uuid":"4b051f94-3f16-4114-83dc-2ea21f6da775","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":25645},{"type":"v1","addr":"172.21.15.134:6835","nonce":25645}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":25645},{"type":"v1","addr":"172.21.15.134:6841","nonce":25645}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6849","nonce":25645},{"type":"v1","addr":"172.21.15.134:6851","nonce":25645}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6844","nonce":25645},{"type":"v1","addr":"172.21.15.134:6846","nonce":25645}]},"public_addr":"172.21.15.134:6835/25645","cluster_addr":"172.21.15.134:6841/25645","heartbeat_back_addr":"172.21.15.134:6851/25645","heartbeat_front_addr":"172.21.15.134:6846/25645","state":["exists","up"]},{"osd":6,"uuid":"7771b052-b691-4ede-9703-fa5a4bc2c6e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6856","nonce":25767},{"type":"v1","addr":"172.21.15.134:6857","nonce":25767}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6858","nonce":25767},{"type":"v1","addr":"172.21.15.134:6859","nonce":25767}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6862","nonce":25767},{"type":"v1","addr":"172.21.15.134:6863","nonce":25767}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6860","nonce":25767},{"type":"v1","addr":"172.21.15.134:6861","nonce":25767}]},"public_addr":"172.21.15.134:6857/25767","cluster_addr":"172.21.15.134:6859/25767","heartbeat_back_addr":"172.21.15.134:6863/25767","heartbeat_front_addr":"172.21.15.134:6861/25767","state":["exists","up"]},{"osd":7,"uuid":"e956d2c2-b6e6-422f-a052-5e7efe34ce26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6864","nonce":25642},{"type":"v1","addr":"172.21.15.134:6866","nonce":25642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6868","nonce":25642},{"type":"v1","addr":"172.21.15.134:6870","nonce":25642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6876","nonce":25642},{"type":"v1","addr":"172.21.15.134:6878","nonce":25642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6872","nonce":25642},{"type":"v1","addr":"172.21.15.134:6874","nonce":25642}]},"public_addr":"172.21.15.134:6866/25642","cluster_addr":"172.21.15.134:6870/25642","heartbeat_back_addr":"172.21.15.134:6878/25642","heartbeat_front_addr":"172.21.15.134:6874/25642","state":["exists","up"]},{"osd":8,"uuid":"1896fcc5-d899-4784-8a2c-11344b1860e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6865","nonce":25694},{"type":"v1","addr":"172.21.15.134:6867","nonce":25694}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6869","nonce":25694},{"type":"v1","addr":"172.21.15.134:6871","nonce":25694}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6877","nonce":25694},{"type":"v1","addr":"172.21.15.134:6879","nonce":25694}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6873","nonce":25694},{"type":"v1","addr":"172.21.15.134:6875","nonce":25694}]},"public_addr":"172.21.15.134:6867/25694","cluster_addr":"172.21.15.134:6871/25694","heartbeat_back_addr":"172.21.15.134:6879/25694","heartbeat_front_addr":"172.21.15.134:6875/25694","state":["exists","up"]},{"osd":9,"uuid":"818002b3-6bbd-4802-95e5-eaef0a009ccb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6833","nonce":25649},{"type":"v1","addr":"172.21.15.134:6836","nonce":25649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6839","nonce":25649},{"type":"v1","addr":"172.21.15.134:6842","nonce":25649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6852","nonce":25649},{"type":"v1","addr":"172.21.15.134:6854","nonce":25649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6845","nonce":25649},{"type":"v1","addr":"172.21.15.134:6848","nonce":25649}]},"public_addr":"172.21.15.134:6836/25649","cluster_addr":"172.21.15.134:6842/25649","heartbeat_back_addr":"172.21.15.134:6854/25649","heartbeat_front_addr":"172.21.15.134:6848/25649","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T10:32:13.629735+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T10:32:13.634602+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T10:32:13.644041+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T10:32:13.431697+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T10:32:13.848559+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T10:32:13.632248+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T10:32:13.676354+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T10:32:14.111210+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T10:32:14.173467+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T10:32:13.436859+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":1,"snaps":[{"begin":2,"length":1}]}],"new_removed_snaps":[{"pool":1,"snaps":[{"begin":2,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-18T10:32:23.154 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-09-18T10:32:23.155 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-18T10:32:23.441 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-18T10:32:23.441 INFO:teuthology.orchestra.run.smithi134.stdout:{"epoch":20,"fsid":"909a0a21-79ad-49e6-bd09-2f76d1ea12ab","created":"2024-09-18T10:32:06.815161+0000","modified":"2024-09-18T10:32:22.466240+0000","last_up_change":"2024-09-18T10:32:15.421636+0000","last_in_change":"2024-09-18T10:32:09.430449+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":10,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2024-09-18T10:32:18.531989+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":20,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"5d600f22-6bed-4106-a48d-b8af006ace47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":25641},{"type":"v1","addr":"172.21.15.134:6837","nonce":25641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6840","nonce":25641},{"type":"v1","addr":"172.21.15.134:6843","nonce":25641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6853","nonce":25641},{"type":"v1","addr":"172.21.15.134:6855","nonce":25641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6847","nonce":25641},{"type":"v1","addr":"172.21.15.134:6850","nonce":25641}]},"public_addr":"172.21.15.134:6837/25641","cluster_addr":"172.21.15.134:6843/25641","heartbeat_back_addr":"172.21.15.134:6855/25641","heartbeat_front_addr":"172.21.15.134:6850/25641","state":["exists","up"]},{"osd":1,"uuid":"17de89fa-8f8b-495e-8fd6-24d62bfbb4de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":25636},{"type":"v1","addr":"172.21.15.134:6813","nonce":25636}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6817","nonce":25636},{"type":"v1","addr":"172.21.15.134:6820","nonce":25636}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6827","nonce":25636},{"type":"v1","addr":"172.21.15.134:6830","nonce":25636}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":25636},{"type":"v1","addr":"172.21.15.134:6825","nonce":25636}]},"public_addr":"172.21.15.134:6813/25636","cluster_addr":"172.21.15.134:6820/25636","heartbeat_back_addr":"172.21.15.134:6830/25636","heartbeat_front_addr":"172.21.15.134:6825/25636","state":["exists","up"]},{"osd":2,"uuid":"19059e59-5e31-4f29-83a3-9b068d17b228","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":25637},{"type":"v1","addr":"172.21.15.134:6801","nonce":25637}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":25637},{"type":"v1","addr":"172.21.15.134:6803","nonce":25637}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":25637},{"type":"v1","addr":"172.21.15.134:6807","nonce":25637}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":25637},{"type":"v1","addr":"172.21.15.134:6805","nonce":25637}]},"public_addr":"172.21.15.134:6801/25637","cluster_addr":"172.21.15.134:6803/25637","heartbeat_back_addr":"172.21.15.134:6807/25637","heartbeat_front_addr":"172.21.15.134:6805/25637","state":["exists","up"]},{"osd":3,"uuid":"636276c9-3bda-4382-95c1-03bb98e80ac3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":25639},{"type":"v1","addr":"172.21.15.134:6814","nonce":25639}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":25639},{"type":"v1","addr":"172.21.15.134:6818","nonce":25639}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":25639},{"type":"v1","addr":"172.21.15.134:6829","nonce":25639}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6821","nonce":25639},{"type":"v1","addr":"172.21.15.134:6824","nonce":25639}]},"public_addr":"172.21.15.134:6814/25639","cluster_addr":"172.21.15.134:6818/25639","heartbeat_back_addr":"172.21.15.134:6829/25639","heartbeat_front_addr":"172.21.15.134:6824/25639","state":["exists","up"]},{"osd":4,"uuid":"b918785a-5612-49ab-9b71-78433ef817f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":25640},{"type":"v1","addr":"172.21.15.134:6809","nonce":25640}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6811","nonce":25640},{"type":"v1","addr":"172.21.15.134:6815","nonce":25640}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":25640},{"type":"v1","addr":"172.21.15.134:6831","nonce":25640}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6819","nonce":25640},{"type":"v1","addr":"172.21.15.134:6823","nonce":25640}]},"public_addr":"172.21.15.134:6809/25640","cluster_addr":"172.21.15.134:6815/25640","heartbeat_back_addr":"172.21.15.134:6831/25640","heartbeat_front_addr":"172.21.15.134:6823/25640","state":["exists","up"]},{"osd":5,"uuid":"4b051f94-3f16-4114-83dc-2ea21f6da775","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":25645},{"type":"v1","addr":"172.21.15.134:6835","nonce":25645}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":25645},{"type":"v1","addr":"172.21.15.134:6841","nonce":25645}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6849","nonce":25645},{"type":"v1","addr":"172.21.15.134:6851","nonce":25645}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6844","nonce":25645},{"type":"v1","addr":"172.21.15.134:6846","nonce":25645}]},"public_addr":"172.21.15.134:6835/25645","cluster_addr":"172.21.15.134:6841/25645","heartbeat_back_addr":"172.21.15.134:6851/25645","heartbeat_front_addr":"172.21.15.134:6846/25645","state":["exists","up"]},{"osd":6,"uuid":"7771b052-b691-4ede-9703-fa5a4bc2c6e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6856","nonce":25767},{"type":"v1","addr":"172.21.15.134:6857","nonce":25767}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6858","nonce":25767},{"type":"v1","addr":"172.21.15.134:6859","nonce":25767}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6862","nonce":25767},{"type":"v1","addr":"172.21.15.134:6863","nonce":25767}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6860","nonce":25767},{"type":"v1","addr":"172.21.15.134:6861","nonce":25767}]},"public_addr":"172.21.15.134:6857/25767","cluster_addr":"172.21.15.134:6859/25767","heartbeat_back_addr":"172.21.15.134:6863/25767","heartbeat_front_addr":"172.21.15.134:6861/25767","state":["exists","up"]},{"osd":7,"uuid":"e956d2c2-b6e6-422f-a052-5e7efe34ce26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6864","nonce":25642},{"type":"v1","addr":"172.21.15.134:6866","nonce":25642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6868","nonce":25642},{"type":"v1","addr":"172.21.15.134:6870","nonce":25642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6876","nonce":25642},{"type":"v1","addr":"172.21.15.134:6878","nonce":25642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6872","nonce":25642},{"type":"v1","addr":"172.21.15.134:6874","nonce":25642}]},"public_addr":"172.21.15.134:6866/25642","cluster_addr":"172.21.15.134:6870/25642","heartbeat_back_addr":"172.21.15.134:6878/25642","heartbeat_front_addr":"172.21.15.134:6874/25642","state":["exists","up"]},{"osd":8,"uuid":"1896fcc5-d899-4784-8a2c-11344b1860e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6865","nonce":25694},{"type":"v1","addr":"172.21.15.134:6867","nonce":25694}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6869","nonce":25694},{"type":"v1","addr":"172.21.15.134:6871","nonce":25694}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6877","nonce":25694},{"type":"v1","addr":"172.21.15.134:6879","nonce":25694}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6873","nonce":25694},{"type":"v1","addr":"172.21.15.134:6875","nonce":25694}]},"public_addr":"172.21.15.134:6867/25694","cluster_addr":"172.21.15.134:6871/25694","heartbeat_back_addr":"172.21.15.134:6879/25694","heartbeat_front_addr":"172.21.15.134:6875/25694","state":["exists","up"]},{"osd":9,"uuid":"818002b3-6bbd-4802-95e5-eaef0a009ccb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6833","nonce":25649},{"type":"v1","addr":"172.21.15.134:6836","nonce":25649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6839","nonce":25649},{"type":"v1","addr":"172.21.15.134:6842","nonce":25649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6852","nonce":25649},{"type":"v1","addr":"172.21.15.134:6854","nonce":25649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6845","nonce":25649},{"type":"v1","addr":"172.21.15.134:6848","nonce":25649}]},"public_addr":"172.21.15.134:6836/25649","cluster_addr":"172.21.15.134:6842/25649","heartbeat_back_addr":"172.21.15.134:6854/25649","heartbeat_front_addr":"172.21.15.134:6848/25649","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T10:32:13.629735+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T10:32:13.634602+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T10:32:13.644041+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T10:32:13.431697+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T10:32:13.848559+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T10:32:13.632248+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T10:32:13.676354+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T10:32:14.111210+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T10:32:14.173467+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T10:32:13.436859+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":1,"snaps":[{"begin":2,"length":1}]}],"new_removed_snaps":[{"pool":1,"snaps":[{"begin":2,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-18T10:32:23.454 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-09-18T10:32:23.454 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-09-18T10:32:23.455 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-09-18T10:32:23.455 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-09-18T10:32:23.456 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-09-18T10:32:23.456 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-09-18T10:32:23.456 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-09-18T10:32:23.457 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-09-18T10:32:23.457 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.8 flush_pg_stats 2024-09-18T10:32:23.458 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.9 flush_pg_stats 2024-09-18T10:32:23.684 INFO:teuthology.orchestra.run.smithi134.stdout:64424509443 2024-09-18T10:32:23.685 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-18T10:32:23.686 INFO:teuthology.orchestra.run.smithi134.stdout:64424509444 2024-09-18T10:32:23.686 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-18T10:32:23.687 INFO:teuthology.orchestra.run.smithi134.stdout:64424509445 2024-09-18T10:32:23.687 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-18T10:32:23.700 INFO:teuthology.orchestra.run.smithi134.stdout:64424509445 2024-09-18T10:32:23.700 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-18T10:32:23.705 INFO:teuthology.orchestra.run.smithi134.stdout:64424509443 2024-09-18T10:32:23.705 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-09-18T10:32:23.712 INFO:teuthology.orchestra.run.smithi134.stdout:64424509444 2024-09-18T10:32:23.712 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-18T10:32:23.717 INFO:teuthology.orchestra.run.smithi134.stdout:64424509443 2024-09-18T10:32:23.717 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-09-18T10:32:23.731 INFO:teuthology.orchestra.run.smithi134.stdout:64424509443 2024-09-18T10:32:23.732 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.9 2024-09-18T10:32:23.737 INFO:teuthology.orchestra.run.smithi134.stdout:64424509443 2024-09-18T10:32:23.737 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-18T10:32:23.738 INFO:teuthology.orchestra.run.smithi134.stdout:64424509444 2024-09-18T10:32:23.738 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.8 2024-09-18T10:32:24.206 INFO:teuthology.orchestra.run.smithi134.stdout:64424509442 2024-09-18T10:32:24.246 INFO:teuthology.orchestra.run.smithi134.stdout:64424509443 2024-09-18T10:32:24.247 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509443 got 64424509442 for osd.7 2024-09-18T10:32:24.247 INFO:teuthology.orchestra.run.smithi134.stdout:64424509443 2024-09-18T10:32:24.248 INFO:teuthology.orchestra.run.smithi134.stdout:64424509442 2024-09-18T10:32:24.261 INFO:teuthology.orchestra.run.smithi134.stdout:64424509444 2024-09-18T10:32:24.263 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509444 got 64424509443 for osd.0 2024-09-18T10:32:24.273 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509443 got 64424509442 for osd.6 2024-09-18T10:32:24.276 INFO:teuthology.orchestra.run.smithi134.stdout:64424509442 2024-09-18T10:32:24.277 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509444 got 64424509443 for osd.1 2024-09-18T10:32:24.283 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509445 got 64424509444 for osd.5 2024-09-18T10:32:24.293 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509443 got 64424509442 for osd.2 2024-09-18T10:32:24.308 INFO:teuthology.orchestra.run.smithi134.stdout:64424509442 2024-09-18T10:32:24.316 INFO:teuthology.orchestra.run.smithi134.stdout:64424509442 2024-09-18T10:32:24.318 INFO:teuthology.orchestra.run.smithi134.stdout:64424509444 2024-09-18T10:32:24.323 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509443 got 64424509442 for osd.4 2024-09-18T10:32:24.332 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509443 got 64424509442 for osd.9 2024-09-18T10:32:24.335 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509445 got 64424509444 for osd.3 2024-09-18T10:32:24.348 INFO:teuthology.orchestra.run.smithi134.stdout:64424509443 2024-09-18T10:32:24.365 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509444 got 64424509443 for osd.8 2024-09-18T10:32:25.248 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-09-18T10:32:25.264 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-18T10:32:25.274 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-09-18T10:32:25.278 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-18T10:32:25.284 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-18T10:32:25.294 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-18T10:32:25.324 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-18T10:32:25.336 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.9 2024-09-18T10:32:25.336 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-18T10:32:25.366 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.8 2024-09-18T10:32:25.770 INFO:teuthology.orchestra.run.smithi134.stdout:64424509444 2024-09-18T10:32:25.789 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509444 got 64424509444 for osd.1 2024-09-18T10:32:25.790 DEBUG:teuthology.parallel:result is None 2024-09-18T10:32:25.793 INFO:teuthology.orchestra.run.smithi134.stdout:64424509443 2024-09-18T10:32:25.799 INFO:teuthology.orchestra.run.smithi134.stdout:64424509445 2024-09-18T10:32:25.816 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509443 got 64424509443 for osd.7 2024-09-18T10:32:25.816 INFO:teuthology.orchestra.run.smithi134.stdout:64424509443 2024-09-18T10:32:25.816 DEBUG:teuthology.parallel:result is None 2024-09-18T10:32:25.829 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509445 got 64424509445 for osd.5 2024-09-18T10:32:25.830 INFO:teuthology.orchestra.run.smithi134.stdout:64424509443 2024-09-18T10:32:25.830 DEBUG:teuthology.parallel:result is None 2024-09-18T10:32:25.837 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509443 got 64424509443 for osd.4 2024-09-18T10:32:25.837 DEBUG:teuthology.parallel:result is None 2024-09-18T10:32:25.841 INFO:teuthology.orchestra.run.smithi134.stdout:64424509443 2024-09-18T10:32:25.849 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509443 got 64424509443 for osd.6 2024-09-18T10:32:25.849 DEBUG:teuthology.parallel:result is None 2024-09-18T10:32:25.853 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509443 got 64424509443 for osd.2 2024-09-18T10:32:25.853 DEBUG:teuthology.parallel:result is None 2024-09-18T10:32:25.876 INFO:teuthology.orchestra.run.smithi134.stdout:64424509444 2024-09-18T10:32:25.890 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509444 got 64424509444 for osd.0 2024-09-18T10:32:25.891 DEBUG:teuthology.parallel:result is None 2024-09-18T10:32:25.891 INFO:teuthology.orchestra.run.smithi134.stdout:64424509443 2024-09-18T10:32:25.902 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509443 got 64424509443 for osd.9 2024-09-18T10:32:25.902 DEBUG:teuthology.parallel:result is None 2024-09-18T10:32:25.905 INFO:teuthology.orchestra.run.smithi134.stdout:64424509445 2024-09-18T10:32:25.917 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509445 got 64424509445 for osd.3 2024-09-18T10:32:25.917 DEBUG:teuthology.parallel:result is None 2024-09-18T10:32:25.949 INFO:teuthology.orchestra.run.smithi134.stdout:64424509444 2024-09-18T10:32:25.962 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509444 got 64424509444 for osd.8 2024-09-18T10:32:25.962 DEBUG:teuthology.parallel:result is None 2024-09-18T10:32:25.962 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-18T10:32:25.962 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-18T10:32:26.272 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-18T10:32:26.275 INFO:teuthology.orchestra.run.smithi134.stderr:dumped all 2024-09-18T10:32:26.284 INFO:teuthology.orchestra.run.smithi134.stdout:{"pg_ready":true,"pg_map":{"version":16,"stamp":"2024-09-18T10:32:25.407609+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":10,"num_per_pool_osds":10,"num_per_pool_omap_osds":9,"kb":1048576000,"kb_used":2975344,"kb_used_data":1748,"kb_used_omap":16,"kb_used_meta":2973423,"kb_avail":1045600656,"statfs":{"total":1073741824000,"available":1070695071744,"internally_reserved":0,"allocated":1789952,"data_stored":561819,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":17040,"internal_metadata":3044785520},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":8,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-18T10:32:21.157119+0000","last_change":"2024-09-18T10:32:21.157119+0000","last_active":"2024-09-18T10:32:21.157119+0000","last_peered":"2024-09-18T10:32:21.157119+0000","last_clean":"2024-09-18T10:32:21.157119+0000","last_became_active":"2024-09-18T10:32:21.156324+0000","last_became_peered":"2024-09-18T10:32:21.156324+0000","last_unstale":"2024-09-18T10:32:21.157119+0000","last_undegraded":"2024-09-18T10:32:21.157119+0000","last_fullsized":"2024-09-18T10:32:21.157119+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T10:32:19.443095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:32:19.443095+0000","last_clean_scrub_stamp":"2024-09-18T10:32:19.443095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T12:49:01.470488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":8,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-18T10:32:21.323918+0000","last_change":"2024-09-18T10:32:21.323918+0000","last_active":"2024-09-18T10:32:21.323918+0000","last_peered":"2024-09-18T10:32:21.323918+0000","last_clean":"2024-09-18T10:32:21.323918+0000","last_became_active":"2024-09-18T10:32:21.323324+0000","last_became_peered":"2024-09-18T10:32:21.323324+0000","last_unstale":"2024-09-18T10:32:21.323918+0000","last_undegraded":"2024-09-18T10:32:21.323918+0000","last_fullsized":"2024-09-18T10:32:21.323918+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T10:32:19.443095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:32:19.443095+0000","last_clean_scrub_stamp":"2024-09-18T10:32:19.443095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T15:50:35.442021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9],"acting":[6,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":8,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-18T10:32:20.464864+0000","last_change":"2024-09-18T10:32:20.464864+0000","last_active":"2024-09-18T10:32:20.464864+0000","last_peered":"2024-09-18T10:32:20.464864+0000","last_clean":"2024-09-18T10:32:20.464864+0000","last_became_active":"2024-09-18T10:32:20.464529+0000","last_became_peered":"2024-09-18T10:32:20.464529+0000","last_unstale":"2024-09-18T10:32:20.464864+0000","last_undegraded":"2024-09-18T10:32:20.464864+0000","last_fullsized":"2024-09-18T10:32:20.464864+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T10:32:19.443095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:32:19.443095+0000","last_clean_scrub_stamp":"2024-09-18T10:32:19.443095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T12:49:01.470488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8],"acting":[4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-18T10:32:20.463941+0000","last_change":"2024-09-18T10:32:20.461559+0000","last_active":"2024-09-18T10:32:20.463941+0000","last_peered":"2024-09-18T10:32:20.463941+0000","last_clean":"2024-09-18T10:32:20.463941+0000","last_became_active":"2024-09-18T10:32:20.460976+0000","last_became_peered":"2024-09-18T10:32:20.460976+0000","last_unstale":"2024-09-18T10:32:20.463941+0000","last_undegraded":"2024-09-18T10:32:20.463941+0000","last_fullsized":"2024-09-18T10:32:20.463941+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T10:32:19.443095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:32:19.443095+0000","last_clean_scrub_stamp":"2024-09-18T10:32:19.443095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T12:49:01.470488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"18'1","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-18T10:32:20.762592+0000","last_change":"2024-09-18T10:32:20.760948+0000","last_active":"2024-09-18T10:32:20.762592+0000","last_peered":"2024-09-18T10:32:20.762592+0000","last_clean":"2024-09-18T10:32:20.762592+0000","last_became_active":"2024-09-18T10:32:20.759131+0000","last_became_peered":"2024-09-18T10:32:20.759131+0000","last_unstale":"2024-09-18T10:32:20.762592+0000","last_undegraded":"2024-09-18T10:32:20.762592+0000","last_fullsized":"2024-09-18T10:32:20.762592+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T10:32:19.443095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:32:19.443095+0000","last_clean_scrub_stamp":"2024-09-18T10:32:19.443095+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T15:50:35.442021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.2","version":"20'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-18T10:32:22.486453+0000","last_change":"2024-09-18T10:32:22.486453+0000","last_active":"2024-09-18T10:32:22.486453+0000","last_peered":"2024-09-18T10:32:22.486453+0000","last_clean":"2024-09-18T10:32:22.486453+0000","last_became_active":"2024-09-18T10:32:21.156409+0000","last_became_peered":"2024-09-18T10:32:21.156409+0000","last_unstale":"2024-09-18T10:32:22.486453+0000","last_undegraded":"2024-09-18T10:32:22.486453+0000","last_fullsized":"2024-09-18T10:32:22.486453+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T10:32:19.443095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:32:19.443095+0000","last_clean_scrub_stamp":"2024-09-18T10:32:19.443095+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:42:55.463754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019475040000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.1","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-18T10:32:22.480954+0000","last_change":"2024-09-18T10:32:22.481392+0000","last_active":"2024-09-18T10:32:22.480954+0000","last_peered":"2024-09-18T10:32:22.480954+0000","last_clean":"2024-09-18T10:32:22.480954+0000","last_became_active":"2024-09-18T10:32:20.461373+0000","last_became_peered":"2024-09-18T10:32:20.461373+0000","last_unstale":"2024-09-18T10:32:22.480954+0000","last_undegraded":"2024-09-18T10:32:22.480954+0000","last_fullsized":"2024-09-18T10:32:22.480954+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T10:32:19.443095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:32:19.443095+0000","last_clean_scrub_stamp":"2024-09-18T10:32:19.443095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:42:55.463754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062722600000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8],"acting":[5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-18T10:32:23.139872+0000","last_change":"2024-09-18T10:32:23.140282+0000","last_active":"2024-09-18T10:32:23.139872+0000","last_peered":"2024-09-18T10:32:23.139872+0000","last_clean":"2024-09-18T10:32:23.139872+0000","last_became_active":"2024-09-18T10:32:20.464817+0000","last_became_peered":"2024-09-18T10:32:20.464817+0000","last_unstale":"2024-09-18T10:32:23.139872+0000","last_undegraded":"2024-09-18T10:32:23.139872+0000","last_fullsized":"2024-09-18T10:32:23.139872+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T10:32:19.443095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:32:19.443095+0000","last_clean_scrub_stamp":"2024-09-18T10:32:19.443095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T19:29:06.426736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056715200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0],"acting":[8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[{"start":"2","length":"1"}]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":9}],"osd_stats":[{"osd":9,"up_from":15,"seq":64424509443,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":297532,"kb_used_data":168,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":104560068,"statfs":{"total":107374182400,"available":107069509632,"internally_reserved":0,"allocated":172032,"data_stored":53107,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,6,7,8],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59999999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3919999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.526}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4770000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.8180000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.8600000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.778}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5620000000000001}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.8500000000000001}]}]},{"osd":8,"up_from":15,"seq":64424509444,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":297532,"kb_used_data":168,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":104560068,"statfs":{"total":107374182400,"available":107069509632,"internally_reserved":0,"allocated":172032,"data_stored":53107,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,6,7,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80200000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.129}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.377}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5269999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2989999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.3109999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6850000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.2130000000000001}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.1800000000000002}]}]},{"osd":7,"up_from":15,"seq":64424509443,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":297552,"kb_used_data":196,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":104560048,"statfs":{"total":107374182400,"available":107069489152,"internally_reserved":0,"allocated":200704,"data_stored":65418,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,6,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84099999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90100000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.103}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.381}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.1379999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.4889999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5389999999999999}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.6389999999999998}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.0680000000000001}]}]},{"osd":6,"up_from":15,"seq":64424509443,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":297528,"kb_used_data":168,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":104560072,"statfs":{"total":107374182400,"available":107069513728,"internally_reserved":0,"allocated":172032,"data_stored":53107,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,7,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.753}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90800000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0269999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7490000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.976}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.1539999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.6440000000000001}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.444}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.2879999999999998}]}]},{"osd":5,"up_from":15,"seq":64424509445,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":297548,"kb_used_data":192,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":104560052,"statfs":{"total":107374182400,"available":107069493248,"internally_reserved":0,"allocated":196608,"data_stored":65399,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,6,7,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0760000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.173}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71699999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.923}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.141}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.613}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.661}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.7080000000000002}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.2130000000000001}]}]},{"osd":4,"up_from":15,"seq":64424509443,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":297532,"kb_used_data":168,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":104560068,"statfs":{"total":107374182400,"available":107069509632,"internally_reserved":0,"allocated":172032,"data_stored":53107,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,5,6,7,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0960000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.875}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.1139999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3069999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.2509999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.2879999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.512}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.173}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.448}]}]},{"osd":3,"up_from":15,"seq":64424509445,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":297536,"kb_used_data":180,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":104560064,"statfs":{"total":107374182400,"available":107069505536,"internally_reserved":0,"allocated":184320,"data_stored":59211,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,4,5,6,7,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73899999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.9259999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.204}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.49}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.0630000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":4.1029999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.1880000000000002}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.5590000000000002}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.1259999999999999}]}]},{"osd":2,"up_from":15,"seq":64424509443,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":104857600,"kb_used":297500,"kb_used_data":144,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":104560100,"statfs":{"total":107374182400,"available":107069542400,"internally_reserved":0,"allocated":147456,"data_stored":40838,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,3,4,5,6,7,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60099999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.651}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72699999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90700000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.597}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.696}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.278}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5509999999999999}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.4220000000000002}]}]},{"osd":1,"up_from":15,"seq":64424509444,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":297532,"kb_used_data":168,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":104560068,"statfs":{"total":107374182400,"available":107069509632,"internally_reserved":0,"allocated":172032,"data_stored":53107,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,2,3,4,5,6,7,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.028}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88200000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":5.7569999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.8700000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.0470000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.1299999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.4900000000000002}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":4.4969999999999999}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.2160000000000002}]}]},{"osd":0,"up_from":15,"seq":64424509444,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":297552,"kb_used_data":196,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":104560048,"statfs":{"total":107374182400,"available":107069489152,"internally_reserved":0,"allocated":200704,"data_stored":65418,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[1,2,3,4,5,6,7,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77400000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.454}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3959999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6339999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7849999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.702}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.004}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.1419999999999999}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.3109999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-18T10:32:26.285 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-18T10:32:26.551 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-18T10:32:26.553 INFO:teuthology.orchestra.run.smithi134.stderr:dumped all 2024-09-18T10:32:26.563 INFO:teuthology.orchestra.run.smithi134.stdout:{"pg_ready":true,"pg_map":{"version":16,"stamp":"2024-09-18T10:32:25.407609+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":10,"num_per_pool_osds":10,"num_per_pool_omap_osds":9,"kb":1048576000,"kb_used":2975344,"kb_used_data":1748,"kb_used_omap":16,"kb_used_meta":2973423,"kb_avail":1045600656,"statfs":{"total":1073741824000,"available":1070695071744,"internally_reserved":0,"allocated":1789952,"data_stored":561819,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":17040,"internal_metadata":3044785520},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":8,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-18T10:32:21.157119+0000","last_change":"2024-09-18T10:32:21.157119+0000","last_active":"2024-09-18T10:32:21.157119+0000","last_peered":"2024-09-18T10:32:21.157119+0000","last_clean":"2024-09-18T10:32:21.157119+0000","last_became_active":"2024-09-18T10:32:21.156324+0000","last_became_peered":"2024-09-18T10:32:21.156324+0000","last_unstale":"2024-09-18T10:32:21.157119+0000","last_undegraded":"2024-09-18T10:32:21.157119+0000","last_fullsized":"2024-09-18T10:32:21.157119+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T10:32:19.443095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:32:19.443095+0000","last_clean_scrub_stamp":"2024-09-18T10:32:19.443095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T12:49:01.470488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":8,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-18T10:32:21.323918+0000","last_change":"2024-09-18T10:32:21.323918+0000","last_active":"2024-09-18T10:32:21.323918+0000","last_peered":"2024-09-18T10:32:21.323918+0000","last_clean":"2024-09-18T10:32:21.323918+0000","last_became_active":"2024-09-18T10:32:21.323324+0000","last_became_peered":"2024-09-18T10:32:21.323324+0000","last_unstale":"2024-09-18T10:32:21.323918+0000","last_undegraded":"2024-09-18T10:32:21.323918+0000","last_fullsized":"2024-09-18T10:32:21.323918+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T10:32:19.443095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:32:19.443095+0000","last_clean_scrub_stamp":"2024-09-18T10:32:19.443095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T15:50:35.442021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9],"acting":[6,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":8,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-18T10:32:20.464864+0000","last_change":"2024-09-18T10:32:20.464864+0000","last_active":"2024-09-18T10:32:20.464864+0000","last_peered":"2024-09-18T10:32:20.464864+0000","last_clean":"2024-09-18T10:32:20.464864+0000","last_became_active":"2024-09-18T10:32:20.464529+0000","last_became_peered":"2024-09-18T10:32:20.464529+0000","last_unstale":"2024-09-18T10:32:20.464864+0000","last_undegraded":"2024-09-18T10:32:20.464864+0000","last_fullsized":"2024-09-18T10:32:20.464864+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T10:32:19.443095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:32:19.443095+0000","last_clean_scrub_stamp":"2024-09-18T10:32:19.443095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T12:49:01.470488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8],"acting":[4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-18T10:32:20.463941+0000","last_change":"2024-09-18T10:32:20.461559+0000","last_active":"2024-09-18T10:32:20.463941+0000","last_peered":"2024-09-18T10:32:20.463941+0000","last_clean":"2024-09-18T10:32:20.463941+0000","last_became_active":"2024-09-18T10:32:20.460976+0000","last_became_peered":"2024-09-18T10:32:20.460976+0000","last_unstale":"2024-09-18T10:32:20.463941+0000","last_undegraded":"2024-09-18T10:32:20.463941+0000","last_fullsized":"2024-09-18T10:32:20.463941+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T10:32:19.443095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:32:19.443095+0000","last_clean_scrub_stamp":"2024-09-18T10:32:19.443095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T12:49:01.470488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"18'1","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2024-09-18T10:32:20.762592+0000","last_change":"2024-09-18T10:32:20.760948+0000","last_active":"2024-09-18T10:32:20.762592+0000","last_peered":"2024-09-18T10:32:20.762592+0000","last_clean":"2024-09-18T10:32:20.762592+0000","last_became_active":"2024-09-18T10:32:20.759131+0000","last_became_peered":"2024-09-18T10:32:20.759131+0000","last_unstale":"2024-09-18T10:32:20.762592+0000","last_undegraded":"2024-09-18T10:32:20.762592+0000","last_fullsized":"2024-09-18T10:32:20.762592+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T10:32:19.443095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:32:19.443095+0000","last_clean_scrub_stamp":"2024-09-18T10:32:19.443095+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T15:50:35.442021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.2","version":"20'2","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-18T10:32:22.486453+0000","last_change":"2024-09-18T10:32:22.486453+0000","last_active":"2024-09-18T10:32:22.486453+0000","last_peered":"2024-09-18T10:32:22.486453+0000","last_clean":"2024-09-18T10:32:22.486453+0000","last_became_active":"2024-09-18T10:32:21.156409+0000","last_became_peered":"2024-09-18T10:32:21.156409+0000","last_unstale":"2024-09-18T10:32:22.486453+0000","last_undegraded":"2024-09-18T10:32:22.486453+0000","last_fullsized":"2024-09-18T10:32:22.486453+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T10:32:19.443095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:32:19.443095+0000","last_clean_scrub_stamp":"2024-09-18T10:32:19.443095+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:42:55.463754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019475040000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.1","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-18T10:32:22.480954+0000","last_change":"2024-09-18T10:32:22.481392+0000","last_active":"2024-09-18T10:32:22.480954+0000","last_peered":"2024-09-18T10:32:22.480954+0000","last_clean":"2024-09-18T10:32:22.480954+0000","last_became_active":"2024-09-18T10:32:20.461373+0000","last_became_peered":"2024-09-18T10:32:20.461373+0000","last_unstale":"2024-09-18T10:32:22.480954+0000","last_undegraded":"2024-09-18T10:32:22.480954+0000","last_fullsized":"2024-09-18T10:32:22.480954+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T10:32:19.443095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:32:19.443095+0000","last_clean_scrub_stamp":"2024-09-18T10:32:19.443095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:42:55.463754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062722600000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8],"acting":[5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"2","length":"1"}]},{"pgid":"1.0","version":"0'0","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-18T10:32:23.139872+0000","last_change":"2024-09-18T10:32:23.140282+0000","last_active":"2024-09-18T10:32:23.139872+0000","last_peered":"2024-09-18T10:32:23.139872+0000","last_clean":"2024-09-18T10:32:23.139872+0000","last_became_active":"2024-09-18T10:32:20.464817+0000","last_became_peered":"2024-09-18T10:32:20.464817+0000","last_unstale":"2024-09-18T10:32:23.139872+0000","last_undegraded":"2024-09-18T10:32:23.139872+0000","last_fullsized":"2024-09-18T10:32:23.139872+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T10:32:19.443095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:32:19.443095+0000","last_clean_scrub_stamp":"2024-09-18T10:32:19.443095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T19:29:06.426736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056715200000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0],"acting":[8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[{"start":"2","length":"1"}]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":9}],"osd_stats":[{"osd":9,"up_from":15,"seq":64424509443,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":297532,"kb_used_data":168,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":104560068,"statfs":{"total":107374182400,"available":107069509632,"internally_reserved":0,"allocated":172032,"data_stored":53107,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,6,7,8],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59999999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3919999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.526}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4770000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.8180000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.8600000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.778}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5620000000000001}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.8500000000000001}]}]},{"osd":8,"up_from":15,"seq":64424509444,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":297532,"kb_used_data":168,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":104560068,"statfs":{"total":107374182400,"available":107069509632,"internally_reserved":0,"allocated":172032,"data_stored":53107,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,6,7,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80200000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.129}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.377}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5269999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2989999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.3109999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6850000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.2130000000000001}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.1800000000000002}]}]},{"osd":7,"up_from":15,"seq":64424509443,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":297552,"kb_used_data":196,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":104560048,"statfs":{"total":107374182400,"available":107069489152,"internally_reserved":0,"allocated":200704,"data_stored":65418,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,6,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84099999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90100000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.103}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.381}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.1379999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.4889999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5389999999999999}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.6389999999999998}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.0680000000000001}]}]},{"osd":6,"up_from":15,"seq":64424509443,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":297528,"kb_used_data":168,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":104560072,"statfs":{"total":107374182400,"available":107069513728,"internally_reserved":0,"allocated":172032,"data_stored":53107,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,7,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.753}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90800000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0269999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7490000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.976}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.1539999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.6440000000000001}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.444}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.2879999999999998}]}]},{"osd":5,"up_from":15,"seq":64424509445,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":297548,"kb_used_data":192,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":104560052,"statfs":{"total":107374182400,"available":107069493248,"internally_reserved":0,"allocated":196608,"data_stored":65399,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,6,7,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0760000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.173}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71699999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.923}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.141}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.613}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.661}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.7080000000000002}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.2130000000000001}]}]},{"osd":4,"up_from":15,"seq":64424509443,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":297532,"kb_used_data":168,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":104560068,"statfs":{"total":107374182400,"available":107069509632,"internally_reserved":0,"allocated":172032,"data_stored":53107,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,5,6,7,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0960000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.875}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.1139999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3069999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.2509999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.2879999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.512}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.173}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.448}]}]},{"osd":3,"up_from":15,"seq":64424509445,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":297536,"kb_used_data":180,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":104560064,"statfs":{"total":107374182400,"available":107069505536,"internally_reserved":0,"allocated":184320,"data_stored":59211,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,4,5,6,7,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73899999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.9259999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.204}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.49}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.0630000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":4.1029999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.1880000000000002}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.5590000000000002}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.1259999999999999}]}]},{"osd":2,"up_from":15,"seq":64424509443,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":104857600,"kb_used":297500,"kb_used_data":144,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":104560100,"statfs":{"total":107374182400,"available":107069542400,"internally_reserved":0,"allocated":147456,"data_stored":40838,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,3,4,5,6,7,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60099999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.651}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72699999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90700000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.597}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.696}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.278}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5509999999999999}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.4220000000000002}]}]},{"osd":1,"up_from":15,"seq":64424509444,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":297532,"kb_used_data":168,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":104560068,"statfs":{"total":107374182400,"available":107069509632,"internally_reserved":0,"allocated":172032,"data_stored":53107,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,2,3,4,5,6,7,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.028}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88200000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":5.7569999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.8700000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.0470000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.1299999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.4900000000000002}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":4.4969999999999999}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.2160000000000002}]}]},{"osd":0,"up_from":15,"seq":64424509444,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":297552,"kb_used_data":196,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":104560048,"statfs":{"total":107374182400,"available":107069489152,"internally_reserved":0,"allocated":200704,"data_stored":65418,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[1,2,3,4,5,6,7,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77400000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.454}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3959999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6339999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7849999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.702}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.004}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.1419999999999999}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.3109999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-18T10:32:26.564 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-18T10:32:26.564 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-09-18T10:32:26.564 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-09-18T10:32:26.564 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-09-18T10:32:26.858 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-18T10:32:26.858 INFO:teuthology.orchestra.run.smithi134.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-09-18T10:32:26.870 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-09-18T10:32:26.871 INFO:teuthology.run_tasks:Running task workunit... 2024-09-18T10:32:26.884 INFO:tasks.workunit:Pulling workunits from ref 7061ea6ca50908d104e9f3979dc615e118d9e153 2024-09-18T10:32:26.884 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-09-18T10:32:26.884 INFO:tasks.workunit:timeout=3h 2024-09-18T10:32:26.884 INFO:tasks.workunit:cleanup=True 2024-09-18T10:32:26.884 DEBUG:teuthology.orchestra.run.smithi134:> stat -- /home/ubuntu/cephtest/mnt.0 2024-09-18T10:32:26.890 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-18T10:32:26.891 INFO:teuthology.orchestra.run.smithi134.stderr:stat: cannot stat '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-09-18T10:32:26.891 DEBUG:teuthology.orchestra.run.smithi134:> mkdir -- /home/ubuntu/cephtest/mnt.0 2024-09-18T10:32:26.897 INFO:tasks.workunit:Created dir /home/ubuntu/cephtest/mnt.0 2024-09-18T10:32:26.897 DEBUG:teuthology.orchestra.run.smithi134:> cd -- /home/ubuntu/cephtest/mnt.0 && mkdir -- client.0 2024-09-18T10:32:26.904 DEBUG:teuthology.orchestra.run.smithi134:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://github.com/ceph/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 7061ea6ca50908d104e9f3979dc615e118d9e153 2024-09-18T10:32:26.911 INFO:tasks.workunit.client.0.smithi134.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-09-18T10:33:19.585 INFO:tasks.workunit.client.0.smithi134.stderr:Note: switching to '7061ea6ca50908d104e9f3979dc615e118d9e153'. 2024-09-18T10:33:19.585 INFO:tasks.workunit.client.0.smithi134.stderr: 2024-09-18T10:33:19.586 INFO:tasks.workunit.client.0.smithi134.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-09-18T10:33:19.586 INFO:tasks.workunit.client.0.smithi134.stderr:changes and commit them, and you can discard any commits you make in this 2024-09-18T10:33:19.586 INFO:tasks.workunit.client.0.smithi134.stderr:state without impacting any branches by switching back to a branch. 2024-09-18T10:33:19.586 INFO:tasks.workunit.client.0.smithi134.stderr: 2024-09-18T10:33:19.586 INFO:tasks.workunit.client.0.smithi134.stderr:If you want to create a new branch to retain commits you create, you may 2024-09-18T10:33:19.586 INFO:tasks.workunit.client.0.smithi134.stderr:do so (now or later) by using -c with the switch command. Example: 2024-09-18T10:33:19.586 INFO:tasks.workunit.client.0.smithi134.stderr: 2024-09-18T10:33:19.587 INFO:tasks.workunit.client.0.smithi134.stderr: git switch -c 2024-09-18T10:33:19.587 INFO:tasks.workunit.client.0.smithi134.stderr: 2024-09-18T10:33:19.587 INFO:tasks.workunit.client.0.smithi134.stderr:Or undo this operation with: 2024-09-18T10:33:19.587 INFO:tasks.workunit.client.0.smithi134.stderr: 2024-09-18T10:33:19.587 INFO:tasks.workunit.client.0.smithi134.stderr: git switch - 2024-09-18T10:33:19.587 INFO:tasks.workunit.client.0.smithi134.stderr: 2024-09-18T10:33:19.588 INFO:tasks.workunit.client.0.smithi134.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-09-18T10:33:19.588 INFO:tasks.workunit.client.0.smithi134.stderr: 2024-09-18T10:33:19.588 INFO:tasks.workunit.client.0.smithi134.stderr:HEAD is now at 7061ea6ca50 Merge branch 'wip-65946-quincy' of https://github.com/amathuria/ceph into wip-bharath11-testing-2024-09-18-1115-quincy 2024-09-18T10:33:19.591 DEBUG:teuthology.orchestra.run.smithi134:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-09-18T10:33:19.652 INFO:tasks.workunit.client.0.smithi134.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-09-18T10:33:19.654 INFO:tasks.workunit.client.0.smithi134.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-09-18T10:33:19.654 INFO:tasks.workunit.client.0.smithi134.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-09-18T10:33:20.538 INFO:tasks.workunit.client.0.smithi134.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-09-18T10:33:20.634 INFO:tasks.workunit.client.0.smithi134.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-09-18T10:33:20.698 INFO:tasks.workunit.client.0.smithi134.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-09-18T10:33:20.700 INFO:tasks.workunit.client.0.smithi134.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-09-18T10:33:20.700 INFO:tasks.workunit.client.0.smithi134.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-09-18T10:33:20.762 INFO:tasks.workunit.client.0.smithi134.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-09-18T10:33:20.768 DEBUG:teuthology.orchestra.run.smithi134:> set -ex 2024-09-18T10:33:20.768 DEBUG:teuthology.orchestra.run.smithi134:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-09-18T10:33:20.776 INFO:tasks.workunit:Running workunits matching rados/test_health_warnings.sh on client.0... 2024-09-18T10:33:20.778 INFO:tasks.workunit:Running workunit rados/test_health_warnings.sh... 2024-09-18T10:33:20.779 DEBUG:teuthology.orchestra.run.smithi134:workunit test rados/test_health_warnings.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=7061ea6ca50908d104e9f3979dc615e118d9e153 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rados/test_health_warnings.sh 2024-09-18T10:33:20.791 INFO:tasks.workunit.client.0.smithi134.stderr:+ crushtool -o crushmap --build --num_osds 10 host straw 2 rack straw 2 row straw 2 root straw 0 2024-09-18T10:33:20.814 INFO:tasks.workunit.client.0.smithi134.stderr:+ ceph osd setcrushmap -i crushmap 2024-09-18T10:33:22.864 INFO:tasks.workunit.client.0.smithi134.stderr:6 2024-09-18T10:33:22.876 INFO:tasks.workunit.client.0.smithi134.stderr:+ ceph osd tree 2024-09-18T10:33:23.250 INFO:tasks.workunit.client.0.smithi134.stdout:ID CLASS WEIGHT TYPE NAME STATUS REWEIGHT PRI-AFF 2024-09-18T10:33:23.250 INFO:tasks.workunit.client.0.smithi134.stdout:-11 10.00000 root root 2024-09-18T10:33:23.250 INFO:tasks.workunit.client.0.smithi134.stdout: -9 8.00000 row row0 2024-09-18T10:33:23.250 INFO:tasks.workunit.client.0.smithi134.stdout: -6 4.00000 rack rack0 2024-09-18T10:33:23.250 INFO:tasks.workunit.client.0.smithi134.stdout: -1 2.00000 host host0 2024-09-18T10:33:23.250 INFO:tasks.workunit.client.0.smithi134.stdout: 0 1.00000 osd.0 up 1.00000 1.00000 2024-09-18T10:33:23.250 INFO:tasks.workunit.client.0.smithi134.stdout: 1 1.00000 osd.1 up 1.00000 1.00000 2024-09-18T10:33:23.250 INFO:tasks.workunit.client.0.smithi134.stdout: -2 2.00000 host host1 2024-09-18T10:33:23.250 INFO:tasks.workunit.client.0.smithi134.stdout: 2 1.00000 osd.2 up 1.00000 1.00000 2024-09-18T10:33:23.250 INFO:tasks.workunit.client.0.smithi134.stdout: 3 1.00000 osd.3 up 1.00000 1.00000 2024-09-18T10:33:23.250 INFO:tasks.workunit.client.0.smithi134.stdout: -7 4.00000 rack rack1 2024-09-18T10:33:23.251 INFO:tasks.workunit.client.0.smithi134.stdout: -3 2.00000 host host2 2024-09-18T10:33:23.251 INFO:tasks.workunit.client.0.smithi134.stdout: 4 1.00000 osd.4 up 1.00000 1.00000 2024-09-18T10:33:23.251 INFO:tasks.workunit.client.0.smithi134.stdout: 5 1.00000 osd.5 up 1.00000 1.00000 2024-09-18T10:33:23.251 INFO:tasks.workunit.client.0.smithi134.stdout: -4 2.00000 host host3 2024-09-18T10:33:23.251 INFO:tasks.workunit.client.0.smithi134.stdout: 6 1.00000 osd.6 up 1.00000 1.00000 2024-09-18T10:33:23.251 INFO:tasks.workunit.client.0.smithi134.stdout: 7 1.00000 osd.7 up 1.00000 1.00000 2024-09-18T10:33:23.251 INFO:tasks.workunit.client.0.smithi134.stdout:-10 2.00000 row row1 2024-09-18T10:33:23.251 INFO:tasks.workunit.client.0.smithi134.stdout: -8 2.00000 rack rack2 2024-09-18T10:33:23.251 INFO:tasks.workunit.client.0.smithi134.stdout: -5 2.00000 host host4 2024-09-18T10:33:23.251 INFO:tasks.workunit.client.0.smithi134.stdout: 8 1.00000 osd.8 up 1.00000 1.00000 2024-09-18T10:33:23.251 INFO:tasks.workunit.client.0.smithi134.stdout: 9 1.00000 osd.9 up 1.00000 1.00000 2024-09-18T10:33:23.262 INFO:tasks.workunit.client.0.smithi134.stderr:+ ceph tell 'osd.*' injectargs --osd_max_markdown_count 1024 --osd_max_markdown_period 1 2024-09-18T10:33:23.407 INFO:tasks.workunit.client.0.smithi134.stdout:osd.0: {} 2024-09-18T10:33:23.447 INFO:tasks.workunit.client.0.smithi134.stdout:osd.1: {} 2024-09-18T10:33:23.488 INFO:tasks.workunit.client.0.smithi134.stdout:osd.2: {} 2024-09-18T10:33:23.527 INFO:tasks.workunit.client.0.smithi134.stdout:osd.3: {} 2024-09-18T10:33:23.565 INFO:tasks.workunit.client.0.smithi134.stdout:osd.4: {} 2024-09-18T10:33:23.597 INFO:tasks.workunit.client.0.smithi134.stdout:osd.5: {} 2024-09-18T10:33:23.632 INFO:tasks.workunit.client.0.smithi134.stdout:osd.6: {} 2024-09-18T10:33:23.666 INFO:tasks.workunit.client.0.smithi134.stdout:osd.7: {} 2024-09-18T10:33:23.697 INFO:tasks.workunit.client.0.smithi134.stdout:osd.8: {} 2024-09-18T10:33:23.738 INFO:tasks.workunit.client.0.smithi134.stdout:osd.9: {} 2024-09-18T10:33:23.749 INFO:tasks.workunit.client.0.smithi134.stderr:+ ceph osd set noout 2024-09-18T10:33:25.870 INFO:tasks.workunit.client.0.smithi134.stderr:noout is set 2024-09-18T10:33:25.883 INFO:tasks.workunit.client.0.smithi134.stderr:+ test_mark_two_osds_same_host_down 2024-09-18T10:33:25.884 INFO:tasks.workunit.client.0.smithi134.stderr:+ ceph osd set noup 2024-09-18T10:33:27.884 INFO:tasks.workunit.client.0.smithi134.stderr:noup is set 2024-09-18T10:33:27.897 INFO:tasks.workunit.client.0.smithi134.stderr:+ ceph osd down osd.0 osd.1 2024-09-18T10:33:28.194 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-18T10:33:28.189+0000 7f71c7940700 -1 mon.a@0(leader).osd e27 definitely_dead 0 2024-09-18T10:33:28.994 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-18T10:33:28.989+0000 7f71c7940700 -1 mon.a@0(leader).osd e28 definitely_dead 0 2024-09-18T10:33:28.998 INFO:tasks.workunit.client.0.smithi134.stderr:osd.0 is already down. osd.1 is already down. 2024-09-18T10:33:29.006 INFO:tasks.workunit.client.0.smithi134.stderr:+ ceph health detail 2024-09-18T10:33:29.477 INFO:tasks.workunit.client.0.smithi134.stdout:HEALTH_WARN noup,noout flag(s) set; 2 osds down; 1 host (2 osds) down 2024-09-18T10:33:29.477 INFO:tasks.workunit.client.0.smithi134.stdout:[WRN] OSDMAP_FLAGS: noup,noout flag(s) set 2024-09-18T10:33:29.477 INFO:tasks.workunit.client.0.smithi134.stdout:[WRN] OSD_DOWN: 2 osds down 2024-09-18T10:33:29.478 INFO:tasks.workunit.client.0.smithi134.stdout: osd.0 (root=root,row=row0,rack=rack0,host=host0) is down 2024-09-18T10:33:29.478 INFO:tasks.workunit.client.0.smithi134.stdout: osd.1 (root=root,row=row0,rack=rack0,host=host0) is down 2024-09-18T10:33:29.478 INFO:tasks.workunit.client.0.smithi134.stdout:[WRN] OSD_HOST_DOWN: 1 host (2 osds) down 2024-09-18T10:33:29.478 INFO:tasks.workunit.client.0.smithi134.stdout: host host0 (root=root,row=row0,rack=rack0) (2 osds) is down 2024-09-18T10:33:29.494 INFO:tasks.workunit.client.0.smithi134.stderr:+ ceph health 2024-09-18T10:33:29.494 INFO:tasks.workunit.client.0.smithi134.stderr:+ grep '1 host' 2024-09-18T10:33:29.917 INFO:tasks.workunit.client.0.smithi134.stdout:HEALTH_WARN noup,noout flag(s) set; 2 osds down; 1 host (2 osds) down 2024-09-18T10:33:29.920 INFO:tasks.workunit.client.0.smithi134.stderr:+ ceph health 2024-09-18T10:33:29.920 INFO:tasks.workunit.client.0.smithi134.stderr:+ grep '2 osds' 2024-09-18T10:33:30.342 INFO:tasks.workunit.client.0.smithi134.stdout:HEALTH_WARN noup,noout flag(s) set; 2 osds down; 1 host (2 osds) down 2024-09-18T10:33:30.343 INFO:tasks.workunit.client.0.smithi134.stderr:+ ceph health detail 2024-09-18T10:33:30.343 INFO:tasks.workunit.client.0.smithi134.stderr:+ grep osd.0 2024-09-18T10:33:30.685 INFO:tasks.ceph.osd.0.smithi134.stderr:2024-09-18T10:33:30.681+0000 7fd4682f9700 -1 osd.0 29 osdmap NOUP flag is set, waiting for it to clear 2024-09-18T10:33:30.806 INFO:tasks.workunit.client.0.smithi134.stdout: osd.0 (root=root,row=row0,rack=rack0,host=host0) is down 2024-09-18T10:33:30.807 INFO:tasks.workunit.client.0.smithi134.stderr:+ ceph health detail 2024-09-18T10:33:30.807 INFO:tasks.workunit.client.0.smithi134.stderr:+ grep osd.1 2024-09-18T10:33:30.894 INFO:tasks.ceph.osd.0.smithi134.stderr:2024-09-18T10:33:30.889+0000 7fd45c083700 -1 osd.0 30 osdmap NOUP flag is set, waiting for it to clear 2024-09-18T10:33:31.307 INFO:tasks.workunit.client.0.smithi134.stdout: osd.1 (root=root,row=row0,rack=rack0,host=host0) is down 2024-09-18T10:33:31.307 INFO:tasks.workunit.client.0.smithi134.stderr:+ ceph osd unset noup 2024-09-18T10:33:31.910 INFO:tasks.ceph.osd.0.smithi134.stderr:2024-09-18T10:33:31.905+0000 7fd4648a3700 -1 osd.0 31 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-18T10:33:32.920 INFO:tasks.workunit.client.0.smithi134.stderr:noup is unset 2024-09-18T10:33:32.931 INFO:tasks.workunit.client.0.smithi134.stderr:+ wait_for_healthy 2024-09-18T10:33:32.932 INFO:tasks.workunit.client.0.smithi134.stderr:+ ceph health 2024-09-18T10:33:32.932 INFO:tasks.workunit.client.0.smithi134.stderr:+ grep down 2024-09-18T10:33:33.372 INFO:tasks.workunit.client.0.smithi134.stdout:HEALTH_WARN noout flag(s) set; 1 osds down; Degraded data redundancy: 1/4 objects degraded (25.000%), 1 pg degraded 2024-09-18T10:33:33.372 INFO:tasks.workunit.client.0.smithi134.stderr:+ sleep 1 2024-09-18T10:33:34.374 INFO:tasks.workunit.client.0.smithi134.stderr:+ ceph health 2024-09-18T10:33:34.374 INFO:tasks.workunit.client.0.smithi134.stderr:+ grep down 2024-09-18T10:33:34.511 INFO:tasks.ceph.osd.1.smithi134.stderr:2024-09-18T10:33:34.505+0000 7ff32090f700 -1 osd.1 33 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-18T10:33:34.833 INFO:tasks.workunit.client.0.smithi134.stdout:HEALTH_WARN noout flag(s) set; 1 osds down; Degraded data redundancy: 2/4 objects degraded (50.000%), 2 pgs degraded 2024-09-18T10:33:34.833 INFO:tasks.workunit.client.0.smithi134.stderr:+ sleep 1 2024-09-18T10:33:35.835 INFO:tasks.workunit.client.0.smithi134.stderr:+ ceph health 2024-09-18T10:33:35.836 INFO:tasks.workunit.client.0.smithi134.stderr:+ grep down 2024-09-18T10:33:36.291 INFO:tasks.workunit.client.0.smithi134.stderr:+ test_mark_two_osds_same_rack_down 2024-09-18T10:33:36.291 INFO:tasks.workunit.client.0.smithi134.stderr:+ ceph osd set noup 2024-09-18T10:33:37.955 INFO:tasks.workunit.client.0.smithi134.stderr:noup is set 2024-09-18T10:33:37.966 INFO:tasks.workunit.client.0.smithi134.stderr:+ ceph osd down osd.8 osd.9 2024-09-18T10:33:38.257 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-18T10:33:38.253+0000 7f71c7940700 -1 mon.a@0(leader).osd e37 definitely_dead 0 2024-09-18T10:33:39.065 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-18T10:33:39.060+0000 7f71c7940700 -1 mon.a@0(leader).osd e38 definitely_dead 0 2024-09-18T10:33:39.069 INFO:tasks.workunit.client.0.smithi134.stderr:osd.8 is already down. osd.9 is already down. 2024-09-18T10:33:39.077 INFO:tasks.workunit.client.0.smithi134.stderr:+ ceph health detail 2024-09-18T10:33:39.415 INFO:tasks.ceph.osd.9.smithi134.stderr:2024-09-18T10:33:39.408+0000 7f20d9052700 -1 osd.9 38 osdmap NOUP flag is set, waiting for it to clear 2024-09-18T10:33:39.573 INFO:tasks.workunit.client.0.smithi134.stdout:HEALTH_WARN noup,noout flag(s) set; 2 osds down; 1 host (2 osds) down; 1 rack (2 osds) down; 1 row (2 osds) down; Degraded data redundancy: 1/4 objects degraded (25.000%), 1 pg degraded 2024-09-18T10:33:39.574 INFO:tasks.workunit.client.0.smithi134.stdout:[WRN] OSDMAP_FLAGS: noup,noout flag(s) set 2024-09-18T10:33:39.574 INFO:tasks.workunit.client.0.smithi134.stdout:[WRN] OSD_DOWN: 2 osds down 2024-09-18T10:33:39.574 INFO:tasks.workunit.client.0.smithi134.stdout: osd.8 (root=root,row=row1,rack=rack2,host=host4) is down 2024-09-18T10:33:39.574 INFO:tasks.workunit.client.0.smithi134.stdout: osd.9 (root=root,row=row1,rack=rack2,host=host4) is down 2024-09-18T10:33:39.574 INFO:tasks.workunit.client.0.smithi134.stdout:[WRN] OSD_HOST_DOWN: 1 host (2 osds) down 2024-09-18T10:33:39.574 INFO:tasks.workunit.client.0.smithi134.stdout: host host4 (root=root,row=row1,rack=rack2) (2 osds) is down 2024-09-18T10:33:39.574 INFO:tasks.workunit.client.0.smithi134.stdout:[WRN] OSD_RACK_DOWN: 1 rack (2 osds) down 2024-09-18T10:33:39.574 INFO:tasks.workunit.client.0.smithi134.stdout: rack rack2 (root=root,row=row1) (2 osds) is down 2024-09-18T10:33:39.574 INFO:tasks.workunit.client.0.smithi134.stdout:[WRN] OSD_ROW_DOWN: 1 row (2 osds) down 2024-09-18T10:33:39.574 INFO:tasks.workunit.client.0.smithi134.stdout: row row1 (root=root) (2 osds) is down 2024-09-18T10:33:39.575 INFO:tasks.workunit.client.0.smithi134.stdout:[WRN] PG_DEGRADED: Degraded data redundancy: 1/4 objects degraded (25.000%), 1 pg degraded 2024-09-18T10:33:39.575 INFO:tasks.workunit.client.0.smithi134.stdout: pg 1.2 is active+undersized+degraded+wait, acting [4] 2024-09-18T10:33:39.584 INFO:tasks.workunit.client.0.smithi134.stderr:+ grep '1 host' 2024-09-18T10:33:39.584 INFO:tasks.workunit.client.0.smithi134.stderr:+ ceph health 2024-09-18T10:33:39.968 INFO:tasks.ceph.osd.9.smithi134.stderr:2024-09-18T10:33:39.964+0000 7f20cb5b7700 -1 osd.9 39 osdmap NOUP flag is set, waiting for it to clear 2024-09-18T10:33:40.004 INFO:tasks.workunit.client.0.smithi134.stdout:HEALTH_WARN noup,noout flag(s) set; 2 osds down; 1 host (2 osds) down; 1 rack (2 osds) down; 1 row (2 osds) down; Degraded data redundancy: 1/4 objects degraded (25.000%), 1 pg degraded 2024-09-18T10:33:40.005 INFO:tasks.workunit.client.0.smithi134.stderr:+ ceph health 2024-09-18T10:33:40.005 INFO:tasks.workunit.client.0.smithi134.stderr:+ grep '1 rack' 2024-09-18T10:33:40.314 INFO:tasks.ceph.osd.8.smithi134.stderr:2024-09-18T10:33:40.308+0000 7fd07b172700 -1 osd.8 39 osdmap NOUP flag is set, waiting for it to clear 2024-09-18T10:33:40.449 INFO:tasks.workunit.client.0.smithi134.stdout:HEALTH_WARN noup,noout flag(s) set; 2 osds down; 1 host (2 osds) down; 1 rack (2 osds) down; 1 row (2 osds) down; Degraded data redundancy: 1/4 objects degraded (25.000%), 1 pg degraded 2024-09-18T10:33:40.450 INFO:tasks.workunit.client.0.smithi134.stderr:+ ceph health 2024-09-18T10:33:40.450 INFO:tasks.workunit.client.0.smithi134.stderr:+ grep '1 row' 2024-09-18T10:33:40.861 INFO:tasks.workunit.client.0.smithi134.stdout:HEALTH_WARN noup,noout flag(s) set; 2 osds down; 1 host (2 osds) down; 1 rack (2 osds) down; 1 row (2 osds) down; Degraded data redundancy: 1/4 objects degraded (25.000%), 1 pg degraded 2024-09-18T10:33:40.861 INFO:tasks.workunit.client.0.smithi134.stderr:+ ceph health 2024-09-18T10:33:40.861 INFO:tasks.workunit.client.0.smithi134.stderr:+ grep '2 osds' 2024-09-18T10:33:41.301 INFO:tasks.workunit.client.0.smithi134.stdout:HEALTH_WARN noup,noout flag(s) set; 2 osds down; 1 host (2 osds) down; 1 rack (2 osds) down; 1 row (2 osds) down; Degraded data redundancy: 1/4 objects degraded (25.000%), 1 pg degraded 2024-09-18T10:33:41.301 INFO:tasks.workunit.client.0.smithi134.stderr:+ ceph health detail 2024-09-18T10:33:41.301 INFO:tasks.workunit.client.0.smithi134.stderr:+ grep osd.8 2024-09-18T10:33:41.773 INFO:tasks.workunit.client.0.smithi134.stdout: osd.8 (root=root,row=row1,rack=rack2,host=host4) is down 2024-09-18T10:33:41.773 INFO:tasks.workunit.client.0.smithi134.stderr:+ ceph health detail 2024-09-18T10:33:41.773 INFO:tasks.workunit.client.0.smithi134.stderr:+ grep osd.9 2024-09-18T10:33:42.227 INFO:tasks.workunit.client.0.smithi134.stdout: osd.9 (root=root,row=row1,rack=rack2,host=host4) is down 2024-09-18T10:33:42.227 INFO:tasks.workunit.client.0.smithi134.stderr:+ ceph osd unset noup 2024-09-18T10:33:42.977 INFO:tasks.ceph.osd.9.smithi134.stderr:2024-09-18T10:33:42.968+0000 7f20d3dd7700 -1 osd.9 40 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-18T10:33:42.977 INFO:tasks.ceph.osd.8.smithi134.stderr:2024-09-18T10:33:42.968+0000 7fd074ef5700 -1 osd.8 40 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-18T10:33:43.985 INFO:tasks.workunit.client.0.smithi134.stderr:noup is unset 2024-09-18T10:33:43.997 INFO:tasks.workunit.client.0.smithi134.stderr:+ wait_for_healthy 2024-09-18T10:33:43.997 INFO:tasks.workunit.client.0.smithi134.stderr:+ grep down 2024-09-18T10:33:43.998 INFO:tasks.workunit.client.0.smithi134.stderr:+ ceph health 2024-09-18T10:33:44.448 INFO:tasks.workunit.client.0.smithi134.stderr:+ test_mark_all_but_last_osds_down 2024-09-18T10:33:44.448 INFO:tasks.workunit.client.0.smithi134.stderr:+ ceph osd set noup 2024-09-18T10:33:46.003 INFO:tasks.workunit.client.0.smithi134.stderr:noup is set 2024-09-18T10:33:46.015 INFO:tasks.workunit.client.0.smithi134.stderr:++ ceph osd ls 2024-09-18T10:33:46.015 INFO:tasks.workunit.client.0.smithi134.stderr:++ sed '$d' 2024-09-18T10:33:46.409 INFO:tasks.workunit.client.0.smithi134.stderr:+ ceph osd down 0 1 2 3 4 5 6 7 8 2024-09-18T10:33:46.701 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-18T10:33:46.696+0000 7f71c7940700 -1 mon.a@0(leader).osd e43 definitely_dead 0 2024-09-18T10:33:47.113 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-18T10:33:47.108+0000 7f71c7940700 -1 mon.a@0(leader).osd e44 definitely_dead 0 2024-09-18T10:33:47.117 INFO:tasks.workunit.client.0.smithi134.stderr:osd.0 is already down. osd.1 is already down. osd.2 is already down. osd.3 is already down. osd.4 is already down. osd.5 is already down. osd.6 is already down. osd.7 is already down. osd.8 is already down. 2024-09-18T10:33:47.126 INFO:tasks.workunit.client.0.smithi134.stderr:+ ceph health detail 2024-09-18T10:33:47.502 INFO:tasks.ceph.osd.3.smithi134.stderr:2024-09-18T10:33:47.496+0000 7fc9dce0c700 -1 osd.3 44 osdmap NOUP flag is set, waiting for it to clear 2024-09-18T10:33:47.567 INFO:tasks.ceph.osd.0.smithi134.stderr:2024-09-18T10:33:47.560+0000 7fd469b1e700 -1 osd.0 44 osdmap NOUP flag is set, waiting for it to clear 2024-09-18T10:33:47.616 INFO:tasks.workunit.client.0.smithi134.stdout:HEALTH_WARN noup,noout flag(s) set; 9 osds down; 4 hosts (8 osds) down; 2 racks (8 osds) down; 1 row (8 osds) down 2024-09-18T10:33:47.616 INFO:tasks.workunit.client.0.smithi134.stdout:[WRN] OSDMAP_FLAGS: noup,noout flag(s) set 2024-09-18T10:33:47.616 INFO:tasks.workunit.client.0.smithi134.stdout:[WRN] OSD_DOWN: 9 osds down 2024-09-18T10:33:47.617 INFO:tasks.workunit.client.0.smithi134.stdout: osd.0 (root=root,row=row0,rack=rack0,host=host0) is down 2024-09-18T10:33:47.617 INFO:tasks.workunit.client.0.smithi134.stdout: osd.1 (root=root,row=row0,rack=rack0,host=host0) is down 2024-09-18T10:33:47.617 INFO:tasks.workunit.client.0.smithi134.stdout: osd.2 (root=root,row=row0,rack=rack0,host=host1) is down 2024-09-18T10:33:47.617 INFO:tasks.workunit.client.0.smithi134.stdout: osd.3 (root=root,row=row0,rack=rack0,host=host1) is down 2024-09-18T10:33:47.617 INFO:tasks.workunit.client.0.smithi134.stdout: osd.4 (root=root,row=row0,rack=rack1,host=host2) is down 2024-09-18T10:33:47.617 INFO:tasks.workunit.client.0.smithi134.stdout: osd.5 (root=root,row=row0,rack=rack1,host=host2) is down 2024-09-18T10:33:47.617 INFO:tasks.workunit.client.0.smithi134.stdout: osd.6 (root=root,row=row0,rack=rack1,host=host3) is down 2024-09-18T10:33:47.617 INFO:tasks.workunit.client.0.smithi134.stdout: osd.7 (root=root,row=row0,rack=rack1,host=host3) is down 2024-09-18T10:33:47.617 INFO:tasks.workunit.client.0.smithi134.stdout: osd.8 (root=root,row=row1,rack=rack2,host=host4) is down 2024-09-18T10:33:47.617 INFO:tasks.workunit.client.0.smithi134.stdout:[WRN] OSD_HOST_DOWN: 4 hosts (8 osds) down 2024-09-18T10:33:47.617 INFO:tasks.workunit.client.0.smithi134.stdout: host host0 (root=root,row=row0,rack=rack0) (2 osds) is down 2024-09-18T10:33:47.617 INFO:tasks.workunit.client.0.smithi134.stdout: host host1 (root=root,row=row0,rack=rack0) (2 osds) is down 2024-09-18T10:33:47.618 INFO:tasks.workunit.client.0.smithi134.stdout: host host2 (root=root,row=row0,rack=rack1) (2 osds) is down 2024-09-18T10:33:47.618 INFO:tasks.workunit.client.0.smithi134.stdout: host host3 (root=root,row=row0,rack=rack1) (2 osds) is down 2024-09-18T10:33:47.618 INFO:tasks.workunit.client.0.smithi134.stdout:[WRN] OSD_RACK_DOWN: 2 racks (8 osds) down 2024-09-18T10:33:47.618 INFO:tasks.workunit.client.0.smithi134.stdout: rack rack0 (root=root,row=row0) (4 osds) is down 2024-09-18T10:33:47.618 INFO:tasks.workunit.client.0.smithi134.stdout: rack rack1 (root=root,row=row0) (4 osds) is down 2024-09-18T10:33:47.618 INFO:tasks.workunit.client.0.smithi134.stdout:[WRN] OSD_ROW_DOWN: 1 row (8 osds) down 2024-09-18T10:33:47.618 INFO:tasks.workunit.client.0.smithi134.stdout: row row0 (root=root) (8 osds) is down 2024-09-18T10:33:47.628 INFO:tasks.workunit.client.0.smithi134.stderr:+ ceph health 2024-09-18T10:33:47.628 INFO:tasks.workunit.client.0.smithi134.stderr:+ grep '1 row' 2024-09-18T10:33:47.715 INFO:tasks.ceph.osd.2.smithi134.stderr:2024-09-18T10:33:47.708+0000 7fcdce716700 -1 osd.2 44 osdmap NOUP flag is set, waiting for it to clear 2024-09-18T10:33:47.918 INFO:tasks.ceph.osd.4.smithi134.stderr:2024-09-18T10:33:47.912+0000 7ff009a34700 -1 osd.4 44 osdmap NOUP flag is set, waiting for it to clear 2024-09-18T10:33:48.011 INFO:tasks.ceph.osd.2.smithi134.stderr:2024-09-18T10:33:48.004+0000 7fcdc047a700 -1 osd.2 45 osdmap NOUP flag is set, waiting for it to clear 2024-09-18T10:33:48.011 INFO:tasks.ceph.osd.3.smithi134.stderr:2024-09-18T10:33:48.004+0000 7fc9cf371700 -1 osd.3 45 osdmap NOUP flag is set, waiting for it to clear 2024-09-18T10:33:48.012 INFO:tasks.ceph.osd.0.smithi134.stderr:2024-09-18T10:33:48.004+0000 7fd45c083700 -1 osd.0 45 osdmap NOUP flag is set, waiting for it to clear 2024-09-18T10:33:48.012 INFO:tasks.ceph.osd.4.smithi134.stderr:2024-09-18T10:33:48.004+0000 7feffcfbd700 -1 osd.4 45 osdmap NOUP flag is set, waiting for it to clear 2024-09-18T10:33:48.043 INFO:tasks.ceph.osd.7.smithi134.stderr:2024-09-18T10:33:48.036+0000 7f2de445e700 -1 osd.7 45 osdmap NOUP flag is set, waiting for it to clear 2024-09-18T10:33:48.089 INFO:tasks.workunit.client.0.smithi134.stdout:HEALTH_WARN noup,noout flag(s) set; 9 osds down; 4 hosts (8 osds) down; 2 racks (8 osds) down; 1 row (8 osds) down 2024-09-18T10:33:48.090 INFO:tasks.workunit.client.0.smithi134.stderr:+ ceph health 2024-09-18T10:33:48.090 INFO:tasks.workunit.client.0.smithi134.stderr:+ grep '2 racks' 2024-09-18T10:33:48.531 INFO:tasks.workunit.client.0.smithi134.stdout:HEALTH_WARN noup,noout flag(s) set; 9 osds down; 4 hosts (8 osds) down; 2 racks (8 osds) down; 1 row (8 osds) down 2024-09-18T10:33:48.532 INFO:tasks.workunit.client.0.smithi134.stderr:+ ceph health 2024-09-18T10:33:48.532 INFO:tasks.workunit.client.0.smithi134.stderr:+ grep '4 hosts' 2024-09-18T10:33:48.736 INFO:tasks.ceph.osd.5.smithi134.stderr:2024-09-18T10:33:48.732+0000 7fc7c3823700 -1 osd.5 45 osdmap NOUP flag is set, waiting for it to clear 2024-09-18T10:33:48.953 INFO:tasks.workunit.client.0.smithi134.stdout:HEALTH_WARN noup,noout flag(s) set; 9 osds down; 4 hosts (8 osds) down; 2 racks (8 osds) down; 1 row (8 osds) down 2024-09-18T10:33:48.954 INFO:tasks.workunit.client.0.smithi134.stderr:+ ceph health 2024-09-18T10:33:48.954 INFO:tasks.workunit.client.0.smithi134.stderr:+ grep '9 osds' 2024-09-18T10:33:49.016 INFO:tasks.ceph.osd.3.smithi134.stderr:2024-09-18T10:33:49.012+0000 7fc9cf371700 -1 osd.3 46 osdmap NOUP flag is set, waiting for it to clear 2024-09-18T10:33:49.018 INFO:tasks.ceph.osd.4.smithi134.stderr:2024-09-18T10:33:49.012+0000 7feffcfbd700 -1 osd.4 46 osdmap NOUP flag is set, waiting for it to clear 2024-09-18T10:33:49.019 INFO:tasks.ceph.osd.0.smithi134.stderr:2024-09-18T10:33:49.012+0000 7fd45c083700 -1 osd.0 46 osdmap NOUP flag is set, waiting for it to clear 2024-09-18T10:33:49.019 INFO:tasks.ceph.osd.5.smithi134.stderr:2024-09-18T10:33:49.012+0000 7fc7b5587700 -1 osd.5 46 osdmap NOUP flag is set, waiting for it to clear 2024-09-18T10:33:49.019 INFO:tasks.ceph.osd.7.smithi134.stderr:2024-09-18T10:33:49.012+0000 7f2dd59c1700 -1 osd.7 46 osdmap NOUP flag is set, waiting for it to clear 2024-09-18T10:33:49.021 INFO:tasks.ceph.osd.2.smithi134.stderr:2024-09-18T10:33:49.016+0000 7fcdc047a700 -1 osd.2 46 osdmap NOUP flag is set, waiting for it to clear 2024-09-18T10:33:49.148 INFO:tasks.ceph.osd.6.smithi134.stderr:2024-09-18T10:33:49.144+0000 7fa9acd27700 -1 osd.6 46 osdmap NOUP flag is set, waiting for it to clear 2024-09-18T10:33:49.359 INFO:tasks.workunit.client.0.smithi134.stdout:HEALTH_WARN noup,noout flag(s) set; 9 osds down; 4 hosts (8 osds) down; 2 racks (8 osds) down; 1 row (8 osds) down 2024-09-18T10:33:49.359 INFO:tasks.workunit.client.0.smithi134.stderr:+ ceph osd unset noup 2024-09-18T10:33:50.025 INFO:tasks.ceph.osd.2.smithi134.stderr:2024-09-18T10:33:50.020+0000 7fcdc8c9a700 -1 osd.2 47 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-18T10:33:50.026 INFO:tasks.ceph.osd.0.smithi134.stderr:2024-09-18T10:33:50.020+0000 7fd4648a3700 -1 osd.0 47 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-18T10:33:50.026 INFO:tasks.ceph.osd.5.smithi134.stderr:2024-09-18T10:33:50.020+0000 7fc7bdda7700 -1 osd.5 47 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-18T10:33:50.028 INFO:tasks.ceph.osd.4.smithi134.stderr:2024-09-18T10:33:50.024+0000 7ff0057dd700 -1 osd.4 47 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-18T10:33:50.028 INFO:tasks.ceph.osd.6.smithi134.stderr:2024-09-18T10:33:50.020+0000 7fa9a72ab700 -1 osd.6 47 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-18T10:33:50.029 INFO:tasks.ceph.osd.3.smithi134.stderr:2024-09-18T10:33:50.024+0000 7fc9d7b91700 -1 osd.3 47 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-18T10:33:50.029 INFO:tasks.ceph.osd.7.smithi134.stderr:2024-09-18T10:33:50.020+0000 7f2dde1e1700 -1 osd.7 47 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-18T10:33:50.225 INFO:tasks.ceph.osd.8.smithi134.stderr:2024-09-18T10:33:50.220+0000 7fd074ef5700 -1 osd.8 47 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-18T10:33:50.323 INFO:tasks.ceph.osd.1.smithi134.stderr:2024-09-18T10:33:50.316+0000 7ff32090f700 -1 osd.1 47 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-18T10:33:51.053 INFO:tasks.workunit.client.0.smithi134.stderr:noup is unset 2024-09-18T10:33:51.063 INFO:tasks.workunit.client.0.smithi134.stderr:+ wait_for_healthy 2024-09-18T10:33:51.063 INFO:tasks.workunit.client.0.smithi134.stderr:+ grep down 2024-09-18T10:33:51.063 INFO:tasks.workunit.client.0.smithi134.stderr:+ ceph health 2024-09-18T10:33:51.514 INFO:tasks.workunit.client.0.smithi134.stderr:+ test_mark_two_osds_same_host_down_with_classes 2024-09-18T10:33:51.515 INFO:tasks.workunit.client.0.smithi134.stderr:+ ceph osd set noup 2024-09-18T10:33:53.063 INFO:tasks.workunit.client.0.smithi134.stderr:noup is set 2024-09-18T10:33:53.076 INFO:tasks.workunit.client.0.smithi134.stderr:+ ceph osd crush set-device-class ssd osd.0 osd.2 osd.4 osd.6 osd.8 2024-09-18T10:33:55.085 INFO:tasks.workunit.client.0.smithi134.stderr:osd.0 already set to class ssd. set-device-class item id 0 name 'osd.0' device_class 'ssd': no change. osd.2 already set to class ssd. set-device-class item id 2 name 'osd.2' device_class 'ssd': no change. osd.4 already set to class ssd. set-device-class item id 4 name 'osd.4' device_class 'ssd': no change. osd.6 already set to class ssd. set-device-class item id 6 name 'osd.6' device_class 'ssd': no change. osd.8 already set to class ssd. set-device-class item id 8 name 'osd.8' device_class 'ssd': no change. set osd(s) to class 'ssd' 2024-09-18T10:33:55.098 INFO:tasks.workunit.client.0.smithi134.stderr:+ ceph osd crush set-device-class hdd osd.1 osd.3 osd.5 osd.7 osd.9 2024-09-18T10:33:57.094 INFO:tasks.workunit.client.0.smithi134.stderr:osd.1 already set to class hdd. set-device-class item id 1 name 'osd.1' device_class 'hdd': no change. osd.3 already set to class hdd. set-device-class item id 3 name 'osd.3' device_class 'hdd': no change. osd.5 already set to class hdd. set-device-class item id 5 name 'osd.5' device_class 'hdd': no change. osd.7 already set to class hdd. set-device-class item id 7 name 'osd.7' device_class 'hdd': no change. osd.9 already set to class hdd. set-device-class item id 9 name 'osd.9' device_class 'hdd': no change. set osd(s) to class 'hdd' 2024-09-18T10:33:57.106 INFO:tasks.workunit.client.0.smithi134.stderr:+ ceph osd down osd.0 osd.1 2024-09-18T10:33:57.401 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-18T10:33:57.396+0000 7f71c7940700 -1 mon.a@0(leader).osd e54 definitely_dead 0 2024-09-18T10:33:58.214 INFO:tasks.ceph.mon.a.smithi134.stderr:2024-09-18T10:33:58.208+0000 7f71c7940700 -1 mon.a@0(leader).osd e55 definitely_dead 0 2024-09-18T10:33:58.218 INFO:tasks.workunit.client.0.smithi134.stderr:osd.0 is already down. osd.1 is already down. 2024-09-18T10:33:58.227 INFO:tasks.workunit.client.0.smithi134.stderr:+ ceph health detail 2024-09-18T10:33:58.714 INFO:tasks.workunit.client.0.smithi134.stdout:HEALTH_WARN noup,noout flag(s) set; 2 osds down; 1 host (2 osds) down 2024-09-18T10:33:58.714 INFO:tasks.workunit.client.0.smithi134.stdout:[WRN] OSDMAP_FLAGS: noup,noout flag(s) set 2024-09-18T10:33:58.714 INFO:tasks.workunit.client.0.smithi134.stdout:[WRN] OSD_DOWN: 2 osds down 2024-09-18T10:33:58.714 INFO:tasks.workunit.client.0.smithi134.stdout: osd.0 (root=root,row=row0,rack=rack0,host=host0) is down 2024-09-18T10:33:58.714 INFO:tasks.workunit.client.0.smithi134.stdout: osd.1 (root=root,row=row0,rack=rack0,host=host0) is down 2024-09-18T10:33:58.714 INFO:tasks.workunit.client.0.smithi134.stdout:[WRN] OSD_HOST_DOWN: 1 host (2 osds) down 2024-09-18T10:33:58.714 INFO:tasks.workunit.client.0.smithi134.stdout: host host0 (root=root,row=row0,rack=rack0) (2 osds) is down 2024-09-18T10:33:58.724 INFO:tasks.workunit.client.0.smithi134.stderr:+ ceph health 2024-09-18T10:33:58.724 INFO:tasks.workunit.client.0.smithi134.stderr:+ grep '1 host' 2024-09-18T10:33:59.161 INFO:tasks.workunit.client.0.smithi134.stdout:HEALTH_WARN noup,noout flag(s) set; 2 osds down; 1 host (2 osds) down 2024-09-18T10:33:59.161 INFO:tasks.workunit.client.0.smithi134.stderr:+ ceph health 2024-09-18T10:33:59.161 INFO:tasks.workunit.client.0.smithi134.stderr:+ grep '2 osds' 2024-09-18T10:33:59.610 INFO:tasks.workunit.client.0.smithi134.stdout:HEALTH_WARN noup,noout flag(s) set; 2 osds down; 1 host (2 osds) down 2024-09-18T10:33:59.610 INFO:tasks.workunit.client.0.smithi134.stderr:+ ceph health detail 2024-09-18T10:33:59.610 INFO:tasks.workunit.client.0.smithi134.stderr:+ grep osd.0 2024-09-18T10:34:00.085 INFO:tasks.workunit.client.0.smithi134.stdout: osd.0 (root=root,row=row0,rack=rack0,host=host0) is down 2024-09-18T10:34:00.085 INFO:tasks.workunit.client.0.smithi134.stderr:+ grep osd.1 2024-09-18T10:34:00.085 INFO:tasks.workunit.client.0.smithi134.stderr:+ ceph health detail 2024-09-18T10:34:00.287 INFO:tasks.ceph.osd.0.smithi134.stderr:2024-09-18T10:34:00.280+0000 7fd4682f9700 -1 osd.0 57 osdmap NOUP flag is set, waiting for it to clear 2024-09-18T10:34:00.560 INFO:tasks.workunit.client.0.smithi134.stdout: osd.1 (root=root,row=row0,rack=rack0,host=host0) is down 2024-09-18T10:34:00.560 INFO:tasks.workunit.client.0.smithi134.stderr:+ ceph osd unset noup 2024-09-18T10:34:01.112 INFO:tasks.ceph.osd.0.smithi134.stderr:2024-09-18T10:34:01.104+0000 7fd4648a3700 -1 osd.0 58 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-18T10:34:01.425 INFO:tasks.ceph.osd.1.smithi134.stderr:2024-09-18T10:34:01.420+0000 7ff32090f700 -1 osd.1 58 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-18T10:34:02.129 INFO:tasks.workunit.client.0.smithi134.stderr:noup is unset 2024-09-18T10:34:02.139 INFO:tasks.workunit.client.0.smithi134.stderr:+ wait_for_healthy 2024-09-18T10:34:02.139 INFO:tasks.workunit.client.0.smithi134.stderr:+ ceph health 2024-09-18T10:34:02.139 INFO:tasks.workunit.client.0.smithi134.stderr:+ grep down 2024-09-18T10:34:02.592 INFO:tasks.workunit.client.0.smithi134.stderr:+ exit 0 2024-09-18T10:34:02.592 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-09-18T10:34:02.593 DEBUG:teuthology.orchestra.run.smithi134:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-09-18T10:34:02.651 INFO:tasks.workunit:Stopping ['rados/test_health_warnings.sh'] on client.0... 2024-09-18T10:34:02.651 DEBUG:teuthology.orchestra.run.smithi134:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-09-18T10:34:02.905 DEBUG:teuthology.parallel:result is None 2024-09-18T10:34:02.905 DEBUG:teuthology.orchestra.run.smithi134:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-09-18T10:34:02.921 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-09-18T10:34:02.921 DEBUG:teuthology.orchestra.run.smithi134:> rmdir -- /home/ubuntu/cephtest/mnt.0 2024-09-18T10:34:02.984 INFO:tasks.workunit:Deleted artificial mount point /home/ubuntu/cephtest/mnt.0/client.0 2024-09-18T10:34:02.984 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-09-18T10:34:02.997 INFO:tasks.ceph.ceph_manager.ceph:Canceling any pending splits or merges... 2024-09-18T10:34:02.997 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-18T10:34:03.298 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-18T10:34:03.298 INFO:teuthology.orchestra.run.smithi134.stdout:{"epoch":60,"fsid":"909a0a21-79ad-49e6-bd09-2f76d1ea12ab","created":"2024-09-18T10:32:06.815161+0000","modified":"2024-09-18T10:34:03.126450+0000","last_up_change":"2024-09-18T10:34:02.112750+0000","last_in_change":"2024-09-18T10:32:09.430449+0000","flags":"noout,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5800064,"flags_set":["noout","pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":10,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":10,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2024-09-18T10:32:18.531989+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":20,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"5d600f22-6bed-4106-a48d-b8af006ace47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":15,"last_clean_end":58,"up_from":59,"up_thru":38,"down_at":55,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":25641},{"type":"v1","addr":"172.21.15.134:6837","nonce":25641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6843","nonce":3025641},{"type":"v1","addr":"172.21.15.134:6859","nonce":3025641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6853","nonce":25641},{"type":"v1","addr":"172.21.15.134:6855","nonce":25641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6847","nonce":25641},{"type":"v1","addr":"172.21.15.134:6850","nonce":25641}]},"public_addr":"172.21.15.134:6837/25641","cluster_addr":"172.21.15.134:6859/3025641","heartbeat_back_addr":"172.21.15.134:6855/25641","heartbeat_front_addr":"172.21.15.134:6850/25641","state":["exists","up"]},{"osd":1,"uuid":"17de89fa-8f8b-495e-8fd6-24d62bfbb4de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":15,"last_clean_end":58,"up_from":59,"up_thru":59,"down_at":55,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":25636},{"type":"v1","addr":"172.21.15.134:6813","nonce":25636}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":3025636},{"type":"v1","addr":"172.21.15.134:6818","nonce":3025636}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6827","nonce":25636},{"type":"v1","addr":"172.21.15.134:6830","nonce":25636}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":25636},{"type":"v1","addr":"172.21.15.134:6825","nonce":25636}]},"public_addr":"172.21.15.134:6813/25636","cluster_addr":"172.21.15.134:6818/3025636","heartbeat_back_addr":"172.21.15.134:6830/25636","heartbeat_front_addr":"172.21.15.134:6825/25636","state":["exists","up"]},{"osd":2,"uuid":"19059e59-5e31-4f29-83a3-9b068d17b228","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":15,"last_clean_end":47,"up_from":48,"up_thru":0,"down_at":44,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":25637},{"type":"v1","addr":"172.21.15.134:6801","nonce":25637}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6869","nonce":1025637},{"type":"v1","addr":"172.21.15.134:6871","nonce":1025637}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":25637},{"type":"v1","addr":"172.21.15.134:6807","nonce":25637}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":25637},{"type":"v1","addr":"172.21.15.134:6805","nonce":25637}]},"public_addr":"172.21.15.134:6801/25637","cluster_addr":"172.21.15.134:6871/1025637","heartbeat_back_addr":"172.21.15.134:6807/25637","heartbeat_front_addr":"172.21.15.134:6805/25637","state":["exists","up"]},{"osd":3,"uuid":"636276c9-3bda-4382-95c1-03bb98e80ac3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":15,"last_clean_end":47,"up_from":48,"up_thru":59,"down_at":44,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":25639},{"type":"v1","addr":"172.21.15.134:6814","nonce":25639}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":1025639},{"type":"v1","addr":"172.21.15.134:6803","nonce":1025639}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":25639},{"type":"v1","addr":"172.21.15.134:6829","nonce":25639}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6821","nonce":25639},{"type":"v1","addr":"172.21.15.134:6824","nonce":25639}]},"public_addr":"172.21.15.134:6814/25639","cluster_addr":"172.21.15.134:6803/1025639","heartbeat_back_addr":"172.21.15.134:6829/25639","heartbeat_front_addr":"172.21.15.134:6824/25639","state":["exists","up"]},{"osd":4,"uuid":"b918785a-5612-49ab-9b71-78433ef817f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":15,"last_clean_end":47,"up_from":48,"up_thru":59,"down_at":44,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":25640},{"type":"v1","addr":"172.21.15.134:6809","nonce":25640}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6882","nonce":1025640},{"type":"v1","addr":"172.21.15.134:6883","nonce":1025640}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":25640},{"type":"v1","addr":"172.21.15.134:6831","nonce":25640}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6819","nonce":25640},{"type":"v1","addr":"172.21.15.134:6823","nonce":25640}]},"public_addr":"172.21.15.134:6809/25640","cluster_addr":"172.21.15.134:6883/1025640","heartbeat_back_addr":"172.21.15.134:6831/25640","heartbeat_front_addr":"172.21.15.134:6823/25640","state":["exists","up"]},{"osd":5,"uuid":"4b051f94-3f16-4114-83dc-2ea21f6da775","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":15,"last_clean_end":47,"up_from":48,"up_thru":48,"down_at":44,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":25645},{"type":"v1","addr":"172.21.15.134:6835","nonce":25645}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6811","nonce":1025645},{"type":"v1","addr":"172.21.15.134:6815","nonce":1025645}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6849","nonce":25645},{"type":"v1","addr":"172.21.15.134:6851","nonce":25645}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6844","nonce":25645},{"type":"v1","addr":"172.21.15.134:6846","nonce":25645}]},"public_addr":"172.21.15.134:6835/25645","cluster_addr":"172.21.15.134:6815/1025645","heartbeat_back_addr":"172.21.15.134:6851/25645","heartbeat_front_addr":"172.21.15.134:6846/25645","state":["exists","up"]},{"osd":6,"uuid":"7771b052-b691-4ede-9703-fa5a4bc2c6e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":15,"last_clean_end":47,"up_from":48,"up_thru":55,"down_at":44,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6856","nonce":25767},{"type":"v1","addr":"172.21.15.134:6857","nonce":25767}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6868","nonce":1025767},{"type":"v1","addr":"172.21.15.134:6870","nonce":1025767}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6862","nonce":25767},{"type":"v1","addr":"172.21.15.134:6863","nonce":25767}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6860","nonce":25767},{"type":"v1","addr":"172.21.15.134:6861","nonce":25767}]},"public_addr":"172.21.15.134:6857/25767","cluster_addr":"172.21.15.134:6870/1025767","heartbeat_back_addr":"172.21.15.134:6863/25767","heartbeat_front_addr":"172.21.15.134:6861/25767","state":["exists","up"]},{"osd":7,"uuid":"e956d2c2-b6e6-422f-a052-5e7efe34ce26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":15,"last_clean_end":47,"up_from":48,"up_thru":48,"down_at":44,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6864","nonce":25642},{"type":"v1","addr":"172.21.15.134:6866","nonce":25642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":1025642},{"type":"v1","addr":"172.21.15.134:6841","nonce":1025642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6876","nonce":25642},{"type":"v1","addr":"172.21.15.134:6878","nonce":25642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6872","nonce":25642},{"type":"v1","addr":"172.21.15.134:6874","nonce":25642}]},"public_addr":"172.21.15.134:6866/25642","cluster_addr":"172.21.15.134:6841/1025642","heartbeat_back_addr":"172.21.15.134:6878/25642","heartbeat_front_addr":"172.21.15.134:6874/25642","state":["exists","up"]},{"osd":8,"uuid":"1896fcc5-d899-4784-8a2c-11344b1860e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":15,"last_clean_end":47,"up_from":48,"up_thru":17,"down_at":44,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6865","nonce":25694},{"type":"v1","addr":"172.21.15.134:6867","nonce":25694}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6858","nonce":2025694},{"type":"v1","addr":"172.21.15.134:6840","nonce":2025694}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6877","nonce":25694},{"type":"v1","addr":"172.21.15.134:6879","nonce":25694}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6873","nonce":25694},{"type":"v1","addr":"172.21.15.134:6875","nonce":25694}]},"public_addr":"172.21.15.134:6867/25694","cluster_addr":"172.21.15.134:6840/2025694","heartbeat_back_addr":"172.21.15.134:6879/25694","heartbeat_front_addr":"172.21.15.134:6875/25694","state":["exists","up"]},{"osd":9,"uuid":"818002b3-6bbd-4802-95e5-eaef0a009ccb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":15,"last_clean_end":40,"up_from":41,"up_thru":59,"down_at":38,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6833","nonce":25649},{"type":"v1","addr":"172.21.15.134:6836","nonce":25649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6817","nonce":1025649},{"type":"v1","addr":"172.21.15.134:6820","nonce":1025649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6852","nonce":25649},{"type":"v1","addr":"172.21.15.134:6854","nonce":25649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6845","nonce":25649},{"type":"v1","addr":"172.21.15.134:6848","nonce":25649}]},"public_addr":"172.21.15.134:6836/25649","cluster_addr":"172.21.15.134:6820/1025649","heartbeat_back_addr":"172.21.15.134:6854/25649","heartbeat_front_addr":"172.21.15.134:6848/25649","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"2024-09-18T10:33:58.088354+0000","laggy_probability":0.65700000524520874,"laggy_interval":1,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T10:32:13.629735+0000","dead_epoch":56},{"osd":1,"down_stamp":"2024-09-18T10:33:58.088354+0000","laggy_probability":0.65700000524520874,"laggy_interval":1,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T10:32:13.634602+0000","dead_epoch":57},{"osd":2,"down_stamp":"2024-09-18T10:33:46.992600+0000","laggy_probability":0.30000001192092896,"laggy_interval":1,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T10:32:13.644041+0000","dead_epoch":44},{"osd":3,"down_stamp":"2024-09-18T10:33:46.992600+0000","laggy_probability":0.30000001192092896,"laggy_interval":1,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T10:32:13.431697+0000","dead_epoch":44},{"osd":4,"down_stamp":"2024-09-18T10:33:46.992600+0000","laggy_probability":0.30000001192092896,"laggy_interval":1,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T10:32:13.848559+0000","dead_epoch":44},{"osd":5,"down_stamp":"2024-09-18T10:33:46.992600+0000","laggy_probability":0.30000001192092896,"laggy_interval":1,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T10:32:13.632248+0000","dead_epoch":44},{"osd":6,"down_stamp":"2024-09-18T10:33:46.992600+0000","laggy_probability":0.30000001192092896,"laggy_interval":1,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T10:32:13.676354+0000","dead_epoch":45},{"osd":7,"down_stamp":"2024-09-18T10:33:46.992600+0000","laggy_probability":0.30000001192092896,"laggy_interval":1,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T10:32:14.111210+0000","dead_epoch":44},{"osd":8,"down_stamp":"2024-09-18T10:33:46.992600+0000","laggy_probability":0.50999999046325684,"laggy_interval":1,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T10:32:14.173467+0000","dead_epoch":47},{"osd":9,"down_stamp":"2024-09-18T10:33:38.944822+0000","laggy_probability":0.30000001192092896,"laggy_interval":1,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T10:32:13.436859+0000","dead_epoch":38}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-18T10:34:03.316 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-18T10:34:03.316 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-18T10:34:03.613 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-18T10:34:03.616 INFO:teuthology.orchestra.run.smithi134.stderr:dumped all 2024-09-18T10:34:03.628 INFO:teuthology.orchestra.run.smithi134.stdout:{"pg_ready":true,"pg_map":{"version":105,"stamp":"2024-09-18T10:34:03.431290+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":12,"acting":12,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":10,"num_per_pool_osds":10,"num_per_pool_omap_osds":8,"kb":1048576000,"kb_used":2979912,"kb_used_data":6308,"kb_used_omap":16,"kb_used_meta":2973423,"kb_avail":1045596088,"statfs":{"total":1073741824000,"available":1070690394112,"internally_reserved":0,"allocated":6459392,"data_stored":3051408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":17040,"internal_metadata":3044785520},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000990"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":45,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-18T10:33:59.097752+0000","last_change":"2024-09-18T10:33:52.036028+0000","last_active":"2024-09-18T10:33:59.097752+0000","last_peered":"2024-09-18T10:33:59.097752+0000","last_clean":"2024-09-18T10:33:59.097752+0000","last_became_active":"2024-09-18T10:33:52.035591+0000","last_became_peered":"2024-09-18T10:33:52.035591+0000","last_unstale":"2024-09-18T10:33:59.097752+0000","last_undegraded":"2024-09-18T10:33:59.097752+0000","last_fullsized":"2024-09-18T10:33:59.097752+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T10:32:19.443095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:32:19.443095+0000","last_clean_scrub_stamp":"2024-09-18T10:32:19.443095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T18:06:23.838644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00038275899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":61,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-18T10:33:59.101190+0000","last_change":"2024-09-18T10:33:52.042125+0000","last_active":"2024-09-18T10:33:59.101190+0000","last_peered":"2024-09-18T10:33:59.101190+0000","last_clean":"2024-09-18T10:33:59.101190+0000","last_became_active":"2024-09-18T10:33:52.041895+0000","last_became_peered":"2024-09-18T10:33:52.041895+0000","last_unstale":"2024-09-18T10:33:59.101190+0000","last_undegraded":"2024-09-18T10:33:59.101190+0000","last_fullsized":"2024-09-18T10:33:59.101190+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T10:32:19.443095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:32:19.443095+0000","last_clean_scrub_stamp":"2024-09-18T10:32:19.443095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T15:18:02.048451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034975400000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":85,"reported_epoch":57,"state":"active+undersized","last_fresh":"2024-09-18T10:34:00.271592+0000","last_change":"2024-09-18T10:34:00.271514+0000","last_active":"2024-09-18T10:34:00.271592+0000","last_peered":"2024-09-18T10:34:00.271592+0000","last_clean":"2024-09-18T10:33:56.981057+0000","last_became_active":"2024-09-18T10:33:59.102191+0000","last_became_peered":"2024-09-18T10:33:59.102191+0000","last_unstale":"2024-09-18T10:34:00.271592+0000","last_undegraded":"2024-09-18T10:34:00.271592+0000","last_fullsized":"2024-09-18T10:33:59.101046+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T10:32:19.443095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:32:19.443095+0000","last_clean_scrub_stamp":"2024-09-18T10:32:19.443095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T13:05:55.380579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048480000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9],"acting":[9],"avail_no_missing":["9"],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":45,"reported_epoch":56,"state":"active+undersized+wait","last_fresh":"2024-09-18T10:33:59.102960+0000","last_change":"2024-09-18T10:33:59.102253+0000","last_active":"2024-09-18T10:33:59.102960+0000","last_peered":"2024-09-18T10:33:59.102960+0000","last_clean":"2024-09-18T10:33:47.951434+0000","last_became_active":"2024-09-18T10:33:59.102253+0000","last_became_peered":"2024-09-18T10:33:59.102253+0000","last_unstale":"2024-09-18T10:33:59.102960+0000","last_undegraded":"2024-09-18T10:33:59.102960+0000","last_fullsized":"2024-09-18T10:33:59.100957+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T10:32:19.443095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:32:19.443095+0000","last_clean_scrub_stamp":"2024-09-18T10:32:19.443095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T19:47:09.963588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6],"acting":[6],"avail_no_missing":["6"],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.3","version":"18'1","reported_seq":70,"reported_epoch":57,"state":"active+undersized+degraded+wait","last_fresh":"2024-09-18T10:34:00.106984+0000","last_change":"2024-09-18T10:34:00.106902+0000","last_active":"2024-09-18T10:34:00.106984+0000","last_peered":"2024-09-18T10:34:00.106984+0000","last_clean":"2024-09-18T10:33:59.061965+0000","last_became_active":"2024-09-18T10:34:00.106902+0000","last_became_peered":"2024-09-18T10:34:00.106902+0000","last_unstale":"2024-09-18T10:34:00.106984+0000","last_undegraded":"2024-09-18T10:34:00.106209+0000","last_fullsized":"2024-09-18T10:34:00.106121+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T10:32:19.443095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:32:19.443095+0000","last_clean_scrub_stamp":"2024-09-18T10:32:19.443095+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T21:39:17.714747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3],"acting":[3],"avail_no_missing":["3"],"object_location_counts":[{"shards":"3","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.2","version":"20'2","reported_seq":73,"reported_epoch":56,"state":"active+undersized+degraded+wait","last_fresh":"2024-09-18T10:33:59.102609+0000","last_change":"2024-09-18T10:33:59.102002+0000","last_active":"2024-09-18T10:33:59.102609+0000","last_peered":"2024-09-18T10:33:59.102609+0000","last_clean":"2024-09-18T10:33:58.228743+0000","last_became_active":"2024-09-18T10:33:59.102002+0000","last_became_peered":"2024-09-18T10:33:59.102002+0000","last_unstale":"2024-09-18T10:33:59.102609+0000","last_undegraded":"2024-09-18T10:33:59.100769+0000","last_fullsized":"2024-09-18T10:33:59.100508+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T10:32:19.443095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:32:19.443095+0000","last_clean_scrub_stamp":"2024-09-18T10:32:19.443095+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T19:07:49.365548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019475040000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4],"acting":[4],"avail_no_missing":["4"],"object_location_counts":[{"shards":"4","objects":1}],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":42,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-18T10:33:59.061961+0000","last_change":"2024-09-18T10:33:52.038532+0000","last_active":"2024-09-18T10:33:59.061961+0000","last_peered":"2024-09-18T10:33:59.061961+0000","last_clean":"2024-09-18T10:33:59.061961+0000","last_became_active":"2024-09-18T10:33:52.038239+0000","last_became_peered":"2024-09-18T10:33:52.038239+0000","last_unstale":"2024-09-18T10:33:59.061961+0000","last_undegraded":"2024-09-18T10:33:59.061961+0000","last_fullsized":"2024-09-18T10:33:59.061961+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T10:32:19.443095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:32:19.443095+0000","last_clean_scrub_stamp":"2024-09-18T10:32:19.443095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T18:34:12.717302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062722600000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":62,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-18T10:33:59.400931+0000","last_change":"2024-09-18T10:33:52.037375+0000","last_active":"2024-09-18T10:33:59.400931+0000","last_peered":"2024-09-18T10:33:59.400931+0000","last_clean":"2024-09-18T10:33:59.400931+0000","last_became_active":"2024-09-18T10:33:52.037028+0000","last_became_peered":"2024-09-18T10:33:52.037028+0000","last_unstale":"2024-09-18T10:33:59.400931+0000","last_undegraded":"2024-09-18T10:33:59.400931+0000","last_fullsized":"2024-09-18T10:33:59.400931+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T10:32:19.443095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:32:19.443095+0000","last_clean_scrub_stamp":"2024-09-18T10:32:19.443095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T19:41:50.364436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9],"acting":[5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":12,"acting":12,"num_store_stats":9}],"osd_stats":[{"osd":9,"up_from":41,"seq":176093659159,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":298004,"kb_used_data":648,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":104559596,"statfs":{"total":107374182400,"available":107069026304,"internally_reserved":0,"allocated":663552,"data_stored":316651,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[2,3,4,5,6,7,8],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 10:33:18 2024","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":1.1930000000000001,"5min":1.1930000000000001,"15min":1.1930000000000001},"last":1.121},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.3029999999999999,"5min":1.3029999999999999,"15min":1.3029999999999999},"last":0.64900000000000002}]},{"osd":1,"last update":"Wed Sep 18 10:33:18 2024","interfaces":[{"interface":"back","average":{"1min":0.873,"5min":0.873,"15min":0.873},"min":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.3919999999999999,"5min":1.3919999999999999,"15min":1.3919999999999999},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":2.4359999999999999,"5min":2.4359999999999999,"15min":2.4359999999999999},"last":1.6379999999999999}]},{"osd":2,"last update":"Wed Sep 18 10:33:18 2024","interfaces":[{"interface":"back","average":{"1min":1.03,"5min":1.03,"15min":1.03},"min":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":1.526,"5min":1.526,"15min":1.526},"last":0.45300000000000001},{"interface":"front","average":{"1min":1.044,"5min":1.044,"15min":1.044},"min":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"max":{"1min":1.591,"5min":1.591,"15min":1.591},"last":0.52000000000000002}]},{"osd":3,"last update":"Wed Sep 18 10:33:18 2024","interfaces":[{"interface":"back","average":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.542,"5min":1.542,"15min":1.542},"last":0.63800000000000001},{"interface":"front","average":{"1min":1.274,"5min":1.274,"15min":1.274},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.8440000000000001,"5min":1.8440000000000001,"15min":1.8440000000000001},"last":0.98099999999999998}]},{"osd":4,"last update":"Wed Sep 18 10:33:18 2024","interfaces":[{"interface":"back","average":{"1min":1.2430000000000001,"5min":1.2430000000000001,"15min":1.2430000000000001},"min":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"max":{"1min":1.8180000000000001,"5min":1.8180000000000001,"15min":1.8180000000000001},"last":0.67300000000000004},{"interface":"front","average":{"1min":1.276,"5min":1.276,"15min":1.276},"min":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"max":{"1min":1.681,"5min":1.681,"15min":1.681},"last":0.96399999999999997}]},{"osd":5,"last update":"Wed Sep 18 10:33:18 2024","interfaces":[{"interface":"back","average":{"1min":1.3140000000000001,"5min":1.3140000000000001,"15min":1.3140000000000001},"min":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"max":{"1min":1.8600000000000001,"5min":1.8600000000000001,"15min":1.8600000000000001},"last":1.0269999999999999},{"interface":"front","average":{"1min":1.3009999999999999,"5min":1.3009999999999999,"15min":1.3009999999999999},"min":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"max":{"1min":1.7090000000000001,"5min":1.7090000000000001,"15min":1.7090000000000001},"last":0.78100000000000003}]},{"osd":6,"last update":"Wed Sep 18 10:33:18 2024","interfaces":[{"interface":"back","average":{"1min":1.3029999999999999,"5min":1.3029999999999999,"15min":1.3029999999999999},"min":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":2.778,"5min":2.778,"15min":2.778},"last":0.86599999999999999},{"interface":"front","average":{"1min":1.345,"5min":1.345,"15min":1.345},"min":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":1.921,"5min":1.921,"15min":1.921},"last":1.075}]},{"osd":7,"last update":"Wed Sep 18 10:33:18 2024","interfaces":[{"interface":"back","average":{"1min":1.3340000000000001,"5min":1.3340000000000001,"15min":1.3340000000000001},"min":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"max":{"1min":1.6830000000000001,"5min":1.6830000000000001,"15min":1.6830000000000001},"last":1.1439999999999999},{"interface":"front","average":{"1min":1.417,"5min":1.417,"15min":1.417},"min":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"max":{"1min":3.0390000000000001,"5min":3.0390000000000001,"15min":3.0390000000000001},"last":0.90600000000000003}]},{"osd":8,"last update":"Wed Sep 18 10:33:18 2024","interfaces":[{"interface":"back","average":{"1min":1.4330000000000001,"5min":1.4330000000000001,"15min":1.4330000000000001},"min":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"max":{"1min":2.8500000000000001,"5min":2.8500000000000001,"15min":2.8500000000000001},"last":0.93999999999999995},{"interface":"front","average":{"1min":1.4079999999999999,"5min":1.4079999999999999,"15min":1.4079999999999999},"min":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"max":{"1min":2.0070000000000001,"5min":2.0070000000000001,"15min":2.0070000000000001},"last":1.1910000000000001}]}]},{"osd":8,"up_from":48,"seq":206158430232,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":104857600,"kb_used":297956,"kb_used_data":600,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":104559644,"statfs":{"total":107374182400,"available":107069075456,"internally_reserved":0,"allocated":614400,"data_stored":285923,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,6,7,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 10:33:16 2024","interfaces":[{"interface":"back","average":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":1.26,"5min":1.26,"15min":1.26},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":1.238,"5min":1.238,"15min":1.238},"last":0.86299999999999999}]},{"osd":1,"last update":"Wed Sep 18 10:33:16 2024","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.751,"15min":0.751},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":1.129,"5min":1.129,"15min":1.129},"last":1.1359999999999999},{"interface":"front","average":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"min":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.2390000000000001,"5min":1.2390000000000001,"15min":1.2390000000000001},"last":0.92600000000000005}]},{"osd":2,"last update":"Wed Sep 18 10:33:16 2024","interfaces":[{"interface":"back","average":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":1.6439999999999999,"5min":1.6439999999999999,"15min":1.6439999999999999},"last":1.238},{"interface":"front","average":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":1.2529999999999999,"5min":1.2529999999999999,"15min":1.2529999999999999},"last":0.82299999999999995}]},{"osd":3,"last update":"Wed Sep 18 10:33:16 2024","interfaces":[{"interface":"back","average":{"1min":1.109,"5min":1.109,"15min":1.109},"min":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"max":{"1min":1.538,"5min":1.538,"15min":1.538},"last":1.458},{"interface":"front","average":{"1min":1.0389999999999999,"5min":1.0389999999999999,"15min":1.0389999999999999},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.4810000000000001,"5min":1.4810000000000001,"15min":1.4810000000000001},"last":1.581}]},{"osd":4,"last update":"Wed Sep 18 10:33:16 2024","interfaces":[{"interface":"back","average":{"1min":1.0620000000000001,"5min":1.0620000000000001,"15min":1.0620000000000001},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.8169999999999999,"5min":1.8169999999999999,"15min":1.8169999999999999},"last":1.917},{"interface":"front","average":{"1min":1.2549999999999999,"5min":1.2549999999999999,"15min":1.2549999999999999},"min":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"max":{"1min":2.1480000000000001,"5min":2.1480000000000001,"15min":2.1480000000000001},"last":1.3600000000000001}]},{"osd":5,"last update":"Wed Sep 18 10:33:16 2024","interfaces":[{"interface":"back","average":{"1min":1.216,"5min":1.216,"15min":1.216},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":2.3109999999999999,"5min":2.3109999999999999,"15min":2.3109999999999999},"last":1.637},{"interface":"front","average":{"1min":1.161,"5min":1.161,"15min":1.161},"min":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":2.0169999999999999,"5min":2.0169999999999999,"15min":2.0169999999999999},"last":1.7110000000000001}]},{"osd":6,"last update":"Wed Sep 18 10:33:16 2024","interfaces":[{"interface":"back","average":{"1min":1.2889999999999999,"5min":1.2889999999999999,"15min":1.2889999999999999},"min":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"max":{"1min":2.1899999999999999,"5min":2.1899999999999999,"15min":2.1899999999999999},"last":1.9470000000000001},{"interface":"front","average":{"1min":1.1799999999999999,"5min":1.1799999999999999,"15min":1.1799999999999999},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":2.069,"5min":2.069,"15min":2.069},"last":1.764}]},{"osd":7,"last update":"Wed Sep 18 10:33:16 2024","interfaces":[{"interface":"back","average":{"1min":1.2589999999999999,"5min":1.2589999999999999,"15min":1.2589999999999999},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":2.2130000000000001,"5min":2.2130000000000001,"15min":2.2130000000000001},"last":1.5129999999999999},{"interface":"front","average":{"1min":1.4039999999999999,"5min":1.4039999999999999,"15min":1.4039999999999999},"min":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"max":{"1min":2.2450000000000001,"5min":2.2450000000000001,"15min":2.2450000000000001},"last":2.0459999999999998}]},{"osd":9,"last update":"Wed Sep 18 10:33:16 2024","interfaces":[{"interface":"back","average":{"1min":1.304,"5min":1.304,"15min":1.304},"min":{"1min":0.63,"5min":0.63,"15min":0.63},"max":{"1min":2.1800000000000002,"5min":2.1800000000000002,"15min":2.1800000000000002},"last":2.0790000000000002},{"interface":"front","average":{"1min":1.3440000000000001,"5min":1.3440000000000001,"15min":1.3440000000000001},"min":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"max":{"1min":2.3969999999999998,"5min":2.3969999999999998,"15min":2.3969999999999998},"last":1.875}]}]},{"osd":7,"up_from":48,"seq":206158430231,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":297968,"kb_used_data":612,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":104559632,"statfs":{"total":107374182400,"available":107069063168,"internally_reserved":0,"allocated":626688,"data_stored":293647,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,6,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 10:33:17 2024","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.202,"5min":1.202,"15min":1.202},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.75,"5min":0.75,"15min":0.75},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":1.486,"5min":1.486,"15min":1.486},"last":0.98799999999999999}]},{"osd":1,"last update":"Wed Sep 18 10:33:17 2024","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.3009999999999999,"5min":1.3009999999999999,"15min":1.3009999999999999},"last":1.085},{"interface":"front","average":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.8480000000000001,"5min":1.8480000000000001,"15min":1.8480000000000001},"last":0.73799999999999999}]},{"osd":2,"last update":"Wed Sep 18 10:33:17 2024","interfaces":[{"interface":"back","average":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":1.5980000000000001,"5min":1.5980000000000001,"15min":1.5980000000000001},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.3440000000000001,"5min":1.3440000000000001,"15min":1.3440000000000001},"last":0.77800000000000002}]},{"osd":3,"last update":"Wed Sep 18 10:33:17 2024","interfaces":[{"interface":"back","average":{"1min":1.1000000000000001,"5min":1.1000000000000001,"15min":1.1000000000000001},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.9319999999999999,"5min":1.9319999999999999,"15min":1.9319999999999999},"last":0.85099999999999998},{"interface":"front","average":{"1min":1.0720000000000001,"5min":1.0720000000000001,"15min":1.0720000000000001},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.8879999999999999,"5min":1.8879999999999999,"15min":1.8879999999999999},"last":0.91400000000000003}]},{"osd":4,"last update":"Wed Sep 18 10:33:17 2024","interfaces":[{"interface":"back","average":{"1min":1.097,"5min":1.097,"15min":1.097},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":2.1379999999999999,"5min":2.1379999999999999,"15min":2.1379999999999999},"last":1.8680000000000001},{"interface":"front","average":{"1min":1.1060000000000001,"5min":1.1060000000000001,"15min":1.1060000000000001},"min":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":2.0110000000000001,"5min":2.0110000000000001,"15min":2.0110000000000001},"last":0.90400000000000003}]},{"osd":5,"last update":"Wed Sep 18 10:33:17 2024","interfaces":[{"interface":"back","average":{"1min":1.194,"5min":1.194,"15min":1.194},"min":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"max":{"1min":2.4889999999999999,"5min":2.4889999999999999,"15min":2.4889999999999999},"last":1.0329999999999999},{"interface":"front","average":{"1min":1.2230000000000001,"5min":1.2230000000000001,"15min":1.2230000000000001},"min":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"max":{"1min":2.4289999999999998,"5min":2.4289999999999998,"15min":2.4289999999999998},"last":2.056}]},{"osd":6,"last update":"Wed Sep 18 10:33:17 2024","interfaces":[{"interface":"back","average":{"1min":1.2769999999999999,"5min":1.2769999999999999,"15min":1.2769999999999999},"min":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"max":{"1min":2.0950000000000002,"5min":2.0950000000000002,"15min":2.0950000000000002},"last":1.341},{"interface":"front","average":{"1min":1.218,"5min":1.218,"15min":1.218},"min":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":1.9450000000000001,"5min":1.9450000000000001,"15min":1.9450000000000001},"last":1.1930000000000001}]},{"osd":8,"last update":"Wed Sep 18 10:33:17 2024","interfaces":[{"interface":"back","average":{"1min":1.2969999999999999,"5min":1.2969999999999999,"15min":1.2969999999999999},"min":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"max":{"1min":2.6389999999999998,"5min":2.6389999999999998,"15min":2.6389999999999998},"last":1.9810000000000001},{"interface":"front","average":{"1min":1.3600000000000001,"5min":1.3600000000000001,"15min":1.3600000000000001},"min":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"max":{"1min":2.1520000000000001,"5min":2.1520000000000001,"15min":2.1520000000000001},"last":1.2789999999999999}]},{"osd":9,"last update":"Wed Sep 18 10:33:17 2024","interfaces":[{"interface":"back","average":{"1min":1.2470000000000001,"5min":1.2470000000000001,"15min":1.2470000000000001},"min":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":2.0680000000000001,"5min":2.0680000000000001,"15min":2.0680000000000001},"last":2.2469999999999999},{"interface":"front","average":{"1min":1.282,"5min":1.282,"15min":1.282},"min":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"max":{"1min":1.8460000000000001,"5min":1.8460000000000001,"15min":1.8460000000000001},"last":2.9929999999999999}]}]},{"osd":6,"up_from":48,"seq":206158430231,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":297988,"kb_used_data":624,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":104559612,"statfs":{"total":107374182400,"available":107069042688,"internally_reserved":0,"allocated":638976,"data_stored":301295,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,7,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 10:33:17 2024","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":1.073,"5min":1.073,"15min":1.073},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":1.3720000000000001,"5min":1.3720000000000001,"15min":1.3720000000000001},"last":0.70499999999999996}]},{"osd":1,"last update":"Wed Sep 18 10:33:17 2024","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.751,"15min":0.751},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":1.6599999999999999,"5min":1.6599999999999999,"15min":1.6599999999999999},"last":1.095},{"interface":"front","average":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":2.0459999999999998,"5min":2.0459999999999998,"15min":2.0459999999999998},"last":0.61399999999999999}]},{"osd":2,"last update":"Wed Sep 18 10:33:17 2024","interfaces":[{"interface":"back","average":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":1.1399999999999999,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":0.77000000000000002},{"interface":"front","average":{"1min":1.052,"5min":1.052,"15min":1.052},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":1.907,"5min":1.907,"15min":1.907},"last":0.92400000000000004}]},{"osd":3,"last update":"Wed Sep 18 10:33:17 2024","interfaces":[{"interface":"back","average":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.7490000000000001,"5min":1.7490000000000001,"15min":1.7490000000000001},"last":0.749},{"interface":"front","average":{"1min":1.1830000000000001,"5min":1.1830000000000001,"15min":1.1830000000000001},"min":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"max":{"1min":1.8680000000000001,"5min":1.8680000000000001,"15min":1.8680000000000001},"last":1.1679999999999999}]},{"osd":4,"last update":"Wed Sep 18 10:33:17 2024","interfaces":[{"interface":"back","average":{"1min":1.2070000000000001,"5min":1.2070000000000001,"15min":1.2070000000000001},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.976,"5min":1.976,"15min":1.976},"last":1.25},{"interface":"front","average":{"1min":1.127,"5min":1.127,"15min":1.127},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":2.544,"5min":2.544,"15min":2.544},"last":1.6200000000000001}]},{"osd":5,"last update":"Wed Sep 18 10:33:17 2024","interfaces":[{"interface":"back","average":{"1min":1.2170000000000001,"5min":1.2170000000000001,"15min":1.2170000000000001},"min":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"max":{"1min":2.1539999999999999,"5min":2.1539999999999999,"15min":2.1539999999999999},"last":1.698},{"interface":"front","average":{"1min":1.296,"5min":1.296,"15min":1.296},"min":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":2.2469999999999999,"5min":2.2469999999999999,"15min":2.2469999999999999},"last":1.5649999999999999}]},{"osd":7,"last update":"Wed Sep 18 10:33:17 2024","interfaces":[{"interface":"back","average":{"1min":1.246,"5min":1.246,"15min":1.246},"min":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":2.6440000000000001,"5min":2.6440000000000001,"15min":2.6440000000000001},"last":1.2769999999999999},{"interface":"front","average":{"1min":1.7609999999999999,"5min":1.7609999999999999,"15min":1.7609999999999999},"min":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"max":{"1min":10.385,"5min":10.385,"15min":10.385},"last":1.633}]},{"osd":8,"last update":"Wed Sep 18 10:33:17 2024","interfaces":[{"interface":"back","average":{"1min":1.3640000000000001,"5min":1.3640000000000001,"15min":1.3640000000000001},"min":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"max":{"1min":2.444,"5min":2.444,"15min":2.444},"last":1.516},{"interface":"front","average":{"1min":1.274,"5min":1.274,"15min":1.274},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":2.714,"5min":2.714,"15min":2.714},"last":1.3580000000000001}]},{"osd":9,"last update":"Wed Sep 18 10:33:17 2024","interfaces":[{"interface":"back","average":{"1min":1.2789999999999999,"5min":1.2789999999999999,"15min":1.2789999999999999},"min":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"max":{"1min":2.2879999999999998,"5min":2.2879999999999998,"15min":2.2879999999999998},"last":1.768},{"interface":"front","average":{"1min":1.3240000000000001,"5min":1.3240000000000001,"15min":1.3240000000000001},"min":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":2.5960000000000001,"5min":2.5960000000000001,"15min":2.5960000000000001},"last":1.669}]}]},{"osd":5,"up_from":48,"seq":206158430233,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":298000,"kb_used_data":636,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":104559600,"statfs":{"total":107374182400,"available":107069030400,"internally_reserved":0,"allocated":651264,"data_stored":308977,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[2,3,4,6,7,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 10:33:18 2024","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.111,"5min":1.111,"15min":1.111},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"min":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.5249999999999999,"5min":1.5249999999999999,"15min":1.5249999999999999},"last":0.63}]},{"osd":1,"last update":"Wed Sep 18 10:33:18 2024","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.173,"5min":1.173,"15min":1.173},"last":0.79600000000000004},{"interface":"front","average":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.8280000000000001,"5min":1.8280000000000001,"15min":1.8280000000000001},"last":0.88500000000000001}]},{"osd":2,"last update":"Wed Sep 18 10:33:18 2024","interfaces":[{"interface":"back","average":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"min":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":1.4159999999999999,"5min":1.4159999999999999,"15min":1.4159999999999999},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"min":{"1min":0.627,"5min":0.627,"15min":0.627},"max":{"1min":2.3849999999999998,"5min":2.3849999999999998,"15min":2.3849999999999998},"last":0.63200000000000001}]},{"osd":3,"last update":"Wed Sep 18 10:33:18 2024","interfaces":[{"interface":"back","average":{"1min":1.079,"5min":1.079,"15min":1.079},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.923,"5min":1.923,"15min":1.923},"last":0.57099999999999995},{"interface":"front","average":{"1min":1.268,"5min":1.268,"15min":1.268},"min":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"max":{"1min":2.073,"5min":2.073,"15min":2.073},"last":0.72799999999999998}]},{"osd":4,"last update":"Wed Sep 18 10:33:18 2024","interfaces":[{"interface":"back","average":{"1min":1.123,"5min":1.123,"15min":1.123},"min":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"max":{"1min":2.141,"5min":2.141,"15min":2.141},"last":0.60699999999999998},{"interface":"front","average":{"1min":1.2490000000000001,"5min":1.2490000000000001,"15min":1.2490000000000001},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":2.3300000000000001,"5min":2.3300000000000001,"15min":2.3300000000000001},"last":1.05}]},{"osd":6,"last update":"Wed Sep 18 10:33:18 2024","interfaces":[{"interface":"back","average":{"1min":1.2769999999999999,"5min":1.2769999999999999,"15min":1.2769999999999999},"min":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"max":{"1min":1.806,"5min":1.806,"15min":1.806},"last":0.81499999999999995},{"interface":"front","average":{"1min":1.274,"5min":1.274,"15min":1.274},"min":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"max":{"1min":2.6120000000000001,"5min":2.6120000000000001,"15min":2.6120000000000001},"last":0.70199999999999996}]},{"osd":7,"last update":"Wed Sep 18 10:33:18 2024","interfaces":[{"interface":"back","average":{"1min":1.3380000000000001,"5min":1.3380000000000001,"15min":1.3380000000000001},"min":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"max":{"1min":2.661,"5min":2.661,"15min":2.661},"last":1.018},{"interface":"front","average":{"1min":1.3819999999999999,"5min":1.3819999999999999,"15min":1.3819999999999999},"min":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"max":{"1min":2.548,"5min":2.548,"15min":2.548},"last":0.89600000000000002}]},{"osd":8,"last update":"Wed Sep 18 10:33:18 2024","interfaces":[{"interface":"back","average":{"1min":1.359,"5min":1.359,"15min":1.359},"min":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"max":{"1min":2.7080000000000002,"5min":2.7080000000000002,"15min":2.7080000000000002},"last":0.96599999999999997},{"interface":"front","average":{"1min":1.327,"5min":1.327,"15min":1.327},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":2.4399999999999999,"5min":2.4399999999999999,"15min":2.4399999999999999},"last":1.169}]},{"osd":9,"last update":"Wed Sep 18 10:33:18 2024","interfaces":[{"interface":"back","average":{"1min":1.3859999999999999,"5min":1.3859999999999999,"15min":1.3859999999999999},"min":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"max":{"1min":2.2130000000000001,"5min":2.2130000000000001,"15min":2.2130000000000001},"last":0.92700000000000005},{"interface":"front","average":{"1min":1.4019999999999999,"5min":1.4019999999999999,"15min":1.4019999999999999},"min":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"max":{"1min":2.782,"5min":2.782,"15min":2.782},"last":1.0860000000000001}]}]},{"osd":4,"up_from":48,"seq":206158430231,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":297992,"kb_used_data":628,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":104559608,"statfs":{"total":107374182400,"available":107069038592,"internally_reserved":0,"allocated":643072,"data_stored":301314,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,5,6,7,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 10:33:16 2024","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":1.0960000000000001,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.745,"5min":0.745,"15min":0.745},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.171,"5min":1.171,"15min":1.171},"last":0.70799999999999996}]},{"osd":1,"last update":"Wed Sep 18 10:33:16 2024","interfaces":[{"interface":"back","average":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":1.1579999999999999,"5min":1.1579999999999999,"15min":1.1579999999999999},"last":1.0820000000000001},{"interface":"front","average":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.583,"5min":1.583,"15min":1.583},"last":0.77400000000000002}]},{"osd":2,"last update":"Wed Sep 18 10:33:16 2024","interfaces":[{"interface":"back","average":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":2.1139999999999999,"5min":2.1139999999999999,"15min":2.1139999999999999},"last":1.165},{"interface":"front","average":{"1min":0.995,"5min":0.995,"15min":0.995},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.5660000000000001,"5min":1.5660000000000001,"15min":1.5660000000000001},"last":0.56100000000000005}]},{"osd":3,"last update":"Wed Sep 18 10:33:16 2024","interfaces":[{"interface":"back","average":{"1min":1.0620000000000001,"5min":1.0620000000000001,"15min":1.0620000000000001},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.6200000000000001,"5min":1.6200000000000001,"15min":1.6200000000000001},"last":0.91400000000000003},{"interface":"front","average":{"1min":1.1699999999999999,"5min":1.1699999999999999,"15min":1.1699999999999999},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.732,"5min":1.732,"15min":1.732},"last":0.876}]},{"osd":5,"last update":"Wed Sep 18 10:33:16 2024","interfaces":[{"interface":"back","average":{"1min":1.159,"5min":1.159,"15min":1.159},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":2.2509999999999999,"5min":2.2509999999999999,"15min":2.2509999999999999},"last":1.0489999999999999},{"interface":"front","average":{"1min":1.1830000000000001,"5min":1.1830000000000001,"15min":1.1830000000000001},"min":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":1.873,"5min":1.873,"15min":1.873},"last":1.0049999999999999}]},{"osd":6,"last update":"Wed Sep 18 10:33:16 2024","interfaces":[{"interface":"back","average":{"1min":1.214,"5min":1.214,"15min":1.214},"min":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"max":{"1min":2.2879999999999998,"5min":2.2879999999999998,"15min":2.2879999999999998},"last":1.232},{"interface":"front","average":{"1min":1.25,"5min":1.25,"15min":1.25},"min":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":2.661,"5min":2.661,"15min":2.661},"last":1.137}]},{"osd":7,"last update":"Wed Sep 18 10:33:16 2024","interfaces":[{"interface":"back","average":{"1min":1.319,"5min":1.319,"15min":1.319},"min":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"max":{"1min":1.895,"5min":1.895,"15min":1.895},"last":1.415},{"interface":"front","average":{"1min":1.2450000000000001,"5min":1.2450000000000001,"15min":1.2450000000000001},"min":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":2.0579999999999998,"5min":2.0579999999999998,"15min":2.0579999999999998},"last":1.6619999999999999}]},{"osd":8,"last update":"Wed Sep 18 10:33:16 2024","interfaces":[{"interface":"back","average":{"1min":1.278,"5min":1.278,"15min":1.278},"min":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":2.173,"5min":2.173,"15min":2.173},"last":1.573},{"interface":"front","average":{"1min":1.3240000000000001,"5min":1.3240000000000001,"15min":1.3240000000000001},"min":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":1.9750000000000001,"5min":1.9750000000000001,"15min":1.9750000000000001},"last":1.375}]},{"osd":9,"last update":"Wed Sep 18 10:33:16 2024","interfaces":[{"interface":"back","average":{"1min":1.327,"5min":1.327,"15min":1.327},"min":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":2.448,"5min":2.448,"15min":2.448},"last":1.4419999999999999},{"interface":"front","average":{"1min":1.389,"5min":1.389,"15min":1.389},"min":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"max":{"1min":2.242,"5min":2.242,"15min":2.242},"last":1.474}]}]},{"osd":3,"up_from":48,"seq":206158430233,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":298000,"kb_used_data":636,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":104559600,"statfs":{"total":107374182400,"available":107069030400,"internally_reserved":0,"allocated":651264,"data_stored":308977,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[2,4,5,6,7,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 10:33:15 2024","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.095,"5min":1.095,"15min":1.095},"last":1.4199999999999999},{"interface":"front","average":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"min":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":2.121,"5min":2.121,"15min":2.121},"last":0.84499999999999997}]},{"osd":1,"last update":"Wed Sep 18 10:33:15 2024","interfaces":[{"interface":"back","average":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":1.9259999999999999,"5min":1.9259999999999999,"15min":1.9259999999999999},"last":1.1140000000000001},{"interface":"front","average":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"min":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":1.2090000000000001,"5min":1.2090000000000001,"15min":1.2090000000000001},"last":0.80800000000000005}]},{"osd":2,"last update":"Wed Sep 18 10:33:15 2024","interfaces":[{"interface":"back","average":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.204,"5min":1.204,"15min":1.204},"last":0.90900000000000003},{"interface":"front","average":{"1min":1.1870000000000001,"5min":1.1870000000000001,"15min":1.1870000000000001},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":3.2040000000000002,"5min":3.2040000000000002,"15min":3.2040000000000002},"last":2.6230000000000002}]},{"osd":4,"last update":"Wed Sep 18 10:33:15 2024","interfaces":[{"interface":"back","average":{"1min":1.071,"5min":1.071,"15min":1.071},"min":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":1.49,"5min":1.49,"15min":1.49},"last":0.82799999999999996},{"interface":"front","average":{"1min":1.266,"5min":1.266,"15min":1.266},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":2.7519999999999998,"5min":2.7519999999999998,"15min":2.7519999999999998},"last":1.016}]},{"osd":5,"last update":"Wed Sep 18 10:33:15 2024","interfaces":[{"interface":"back","average":{"1min":1.1830000000000001,"5min":1.1830000000000001,"15min":1.1830000000000001},"min":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"max":{"1min":2.0630000000000002,"5min":2.0630000000000002,"15min":2.0630000000000002},"last":0.71299999999999997},{"interface":"front","average":{"1min":1.1259999999999999,"5min":1.1259999999999999,"15min":1.1259999999999999},"min":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"max":{"1min":1.5149999999999999,"5min":1.5149999999999999,"15min":1.5149999999999999},"last":1.115}]},{"osd":6,"last update":"Wed Sep 18 10:33:15 2024","interfaces":[{"interface":"back","average":{"1min":1.3620000000000001,"5min":1.3620000000000001,"15min":1.3620000000000001},"min":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":4.1029999999999998,"5min":4.1029999999999998,"15min":4.1029999999999998},"last":0.97699999999999998},{"interface":"front","average":{"1min":1.268,"5min":1.268,"15min":1.268},"min":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"max":{"1min":2.2789999999999999,"5min":2.2789999999999999,"15min":2.2789999999999999},"last":1.427}]},{"osd":7,"last update":"Wed Sep 18 10:33:15 2024","interfaces":[{"interface":"back","average":{"1min":1.2010000000000001,"5min":1.2010000000000001,"15min":1.2010000000000001},"min":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"max":{"1min":2.1880000000000002,"5min":2.1880000000000002,"15min":2.1880000000000002},"last":2.7589999999999999},{"interface":"front","average":{"1min":1.282,"5min":1.282,"15min":1.282},"min":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"max":{"1min":3.0409999999999999,"5min":3.0409999999999999,"15min":3.0409999999999999},"last":1.2}]},{"osd":8,"last update":"Wed Sep 18 10:33:15 2024","interfaces":[{"interface":"back","average":{"1min":1.5109999999999999,"5min":1.5109999999999999,"15min":1.5109999999999999},"min":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"max":{"1min":3.5590000000000002,"5min":3.5590000000000002,"15min":3.5590000000000002},"last":1.3360000000000001},{"interface":"front","average":{"1min":1.3540000000000001,"5min":1.3540000000000001,"15min":1.3540000000000001},"min":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"max":{"1min":2.3919999999999999,"5min":2.3919999999999999,"15min":2.3919999999999999},"last":1.4850000000000001}]},{"osd":9,"last update":"Wed Sep 18 10:33:15 2024","interfaces":[{"interface":"back","average":{"1min":1.379,"5min":1.379,"15min":1.379},"min":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"max":{"1min":3.1259999999999999,"5min":3.1259999999999999,"15min":3.1259999999999999},"last":1.522},{"interface":"front","average":{"1min":1.5109999999999999,"5min":1.5109999999999999,"15min":1.5109999999999999},"min":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"max":{"1min":3.6339999999999999,"5min":3.6339999999999999,"15min":3.6339999999999999},"last":1.6020000000000001}]}]},{"osd":2,"up_from":48,"seq":206158430231,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":104857600,"kb_used":297996,"kb_used_data":636,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":104559604,"statfs":{"total":107374182400,"available":107069034496,"internally_reserved":0,"allocated":651264,"data_stored":308977,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,3,4,5,6,7,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 10:33:18 2024","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1.109,"5min":1.109,"15min":1.109},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":1.5029999999999999,"5min":1.5029999999999999,"15min":1.5029999999999999},"last":0.69299999999999995}]},{"osd":1,"last update":"Wed Sep 18 10:33:18 2024","interfaces":[{"interface":"back","average":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"min":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.651,"5min":1.651,"15min":1.651},"last":0.71799999999999997},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":1.1559999999999999,"5min":1.1559999999999999,"15min":1.1559999999999999},"last":1.1599999999999999}]},{"osd":3,"last update":"Wed Sep 18 10:33:18 2024","interfaces":[{"interface":"back","average":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":1.2869999999999999,"5min":1.2869999999999999,"15min":1.2869999999999999},"last":0.92800000000000005},{"interface":"front","average":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.7390000000000001,"5min":1.7390000000000001,"15min":1.7390000000000001},"last":0.83899999999999997}]},{"osd":4,"last update":"Wed Sep 18 10:33:18 2024","interfaces":[{"interface":"back","average":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"min":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":1.333,"5min":1.333,"15min":1.333},"last":1.329},{"interface":"front","average":{"1min":1.151,"5min":1.151,"15min":1.151},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.7869999999999999,"5min":1.7869999999999999,"15min":1.7869999999999999},"last":0.89300000000000002}]},{"osd":5,"last update":"Wed Sep 18 10:33:18 2024","interfaces":[{"interface":"back","average":{"1min":1.1100000000000001,"5min":1.1100000000000001,"15min":1.1100000000000001},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.597,"5min":1.597,"15min":1.597},"last":1.2909999999999999},{"interface":"front","average":{"1min":1.246,"5min":1.246,"15min":1.246},"min":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"max":{"1min":1.873,"5min":1.873,"15min":1.873},"last":1.0029999999999999}]},{"osd":6,"last update":"Wed Sep 18 10:33:18 2024","interfaces":[{"interface":"back","average":{"1min":1.0649999999999999,"5min":1.0649999999999999,"15min":1.0649999999999999},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":1.696,"5min":1.696,"15min":1.696},"last":1.6259999999999999},{"interface":"front","average":{"1min":1.1479999999999999,"5min":1.1479999999999999,"15min":1.1479999999999999},"min":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"max":{"1min":1.5289999999999999,"5min":1.5289999999999999,"15min":1.5289999999999999},"last":1.605}]},{"osd":7,"last update":"Wed Sep 18 10:33:18 2024","interfaces":[{"interface":"back","average":{"1min":1.268,"5min":1.268,"15min":1.268},"min":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":1.9350000000000001,"5min":1.9350000000000001,"15min":1.9350000000000001},"last":1.492},{"interface":"front","average":{"1min":1.244,"5min":1.244,"15min":1.244},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":2.6419999999999999,"5min":2.6419999999999999,"15min":2.6419999999999999},"last":1.534}]},{"osd":8,"last update":"Wed Sep 18 10:33:18 2024","interfaces":[{"interface":"back","average":{"1min":1.306,"5min":1.306,"15min":1.306},"min":{"1min":1,"5min":1,"15min":1},"max":{"1min":1.613,"5min":1.613,"15min":1.613},"last":1.093},{"interface":"front","average":{"1min":1.278,"5min":1.278,"15min":1.278},"min":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":1.9830000000000001,"5min":1.9830000000000001,"15min":1.9830000000000001},"last":1.7}]},{"osd":9,"last update":"Wed Sep 18 10:33:18 2024","interfaces":[{"interface":"back","average":{"1min":1.2210000000000001,"5min":1.2210000000000001,"15min":1.2210000000000001},"min":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"max":{"1min":2.4220000000000002,"5min":2.4220000000000002,"15min":2.4220000000000002},"last":1.5720000000000001},{"interface":"front","average":{"1min":1.337,"5min":1.337,"15min":1.337},"min":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"max":{"1min":2.1349999999999998,"5min":2.1349999999999998,"15min":2.1349999999999998},"last":1.0609999999999999}]}]},{"osd":1,"up_from":0,"seq":24,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":298008,"kb_used_data":652,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":104559592,"statfs":{"total":107374182400,"available":107069022208,"internally_reserved":0,"allocated":667648,"data_stored":316670,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[2,3,4,5,6,7,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 10:33:17 2024","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":1.0980000000000001,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.89900000000000002},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"min":{"1min":0.248,"5min":0.248,"15min":0.248},"max":{"1min":1.3,"5min":1.3,"15min":1.3},"last":0.58699999999999997}]},{"osd":2,"last update":"Wed Sep 18 10:33:17 2024","interfaces":[{"interface":"back","average":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.206,"5min":1.206,"15min":1.206},"last":1.1020000000000001},{"interface":"front","average":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":1.54,"5min":1.54,"15min":1.54},"last":0.83199999999999996}]},{"osd":3,"last update":"Wed Sep 18 10:33:17 2024","interfaces":[{"interface":"back","average":{"1min":1.129,"5min":1.129,"15min":1.129},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":5.7569999999999997,"5min":5.7569999999999997,"15min":5.7569999999999997},"last":1.139},{"interface":"front","average":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":2.9830000000000001,"5min":2.9830000000000001,"15min":2.9830000000000001},"last":0.999}]},{"osd":4,"last update":"Wed Sep 18 10:33:17 2024","interfaces":[{"interface":"back","average":{"1min":1.1000000000000001,"5min":1.1000000000000001,"15min":1.1000000000000001},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":3.8700000000000001,"5min":3.8700000000000001,"15min":3.8700000000000001},"last":1.1899999999999999},{"interface":"front","average":{"1min":1.1359999999999999,"5min":1.1359999999999999,"15min":1.1359999999999999},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":2.6520000000000001,"5min":2.6520000000000001,"15min":2.6520000000000001},"last":1.296}]},{"osd":5,"last update":"Wed Sep 18 10:33:17 2024","interfaces":[{"interface":"back","average":{"1min":1.1160000000000001,"5min":1.1160000000000001,"15min":1.1160000000000001},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":2.0470000000000002,"5min":2.0470000000000002,"15min":2.0470000000000002},"last":1.21},{"interface":"front","average":{"1min":1.0920000000000001,"5min":1.0920000000000001,"15min":1.0920000000000001},"min":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"max":{"1min":1.6679999999999999,"5min":1.6679999999999999,"15min":1.6679999999999999},"last":1.4059999999999999}]},{"osd":6,"last update":"Wed Sep 18 10:33:17 2024","interfaces":[{"interface":"back","average":{"1min":1.3280000000000001,"5min":1.3280000000000001,"15min":1.3280000000000001},"min":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":3.1299999999999999,"5min":3.1299999999999999,"15min":3.1299999999999999},"last":1.5489999999999999},{"interface":"front","average":{"1min":1.123,"5min":1.123,"15min":1.123},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.9390000000000001,"5min":1.9390000000000001,"15min":1.9390000000000001},"last":1.3939999999999999}]},{"osd":7,"last update":"Wed Sep 18 10:33:17 2024","interfaces":[{"interface":"back","average":{"1min":1.1060000000000001,"5min":1.1060000000000001,"15min":1.1060000000000001},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":2.4900000000000002,"5min":2.4900000000000002,"15min":2.4900000000000002},"last":1.4710000000000001},{"interface":"front","average":{"1min":1.3120000000000001,"5min":1.3120000000000001,"15min":1.3120000000000001},"min":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":3.2810000000000001,"5min":3.2810000000000001,"15min":3.2810000000000001},"last":1.5109999999999999}]},{"osd":8,"last update":"Wed Sep 18 10:33:17 2024","interfaces":[{"interface":"back","average":{"1min":1.258,"5min":1.258,"15min":1.258},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":4.4969999999999999,"5min":4.4969999999999999,"15min":4.4969999999999999},"last":1.4850000000000001},{"interface":"front","average":{"1min":1.1419999999999999,"5min":1.1419999999999999,"15min":1.1419999999999999},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":2.1629999999999998,"5min":2.1629999999999998,"15min":2.1629999999999998},"last":1.569}]},{"osd":9,"last update":"Wed Sep 18 10:33:17 2024","interfaces":[{"interface":"back","average":{"1min":1.4419999999999999,"5min":1.4419999999999999,"15min":1.4419999999999999},"min":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"max":{"1min":3.2160000000000002,"5min":3.2160000000000002,"15min":3.2160000000000002},"last":1.601},{"interface":"front","average":{"1min":1.2410000000000001,"5min":1.2410000000000001,"15min":1.2410000000000001},"min":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"max":{"1min":1.9430000000000001,"5min":1.9430000000000001,"15min":1.9430000000000001},"last":1.619}]}]},{"osd":0,"up_from":0,"seq":24,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":298000,"kb_used_data":636,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":104559600,"statfs":{"total":107374182400,"available":107069030400,"internally_reserved":0,"allocated":651264,"data_stored":308977,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[2,3,4,5,6,7,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Sep 18 10:33:15 2024","interfaces":[{"interface":"back","average":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":1.5880000000000001,"5min":1.5880000000000001,"15min":1.5880000000000001},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.917,"5min":1.917,"15min":1.917},"last":1.0600000000000001}]},{"osd":2,"last update":"Wed Sep 18 10:33:15 2024","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.746,"15min":0.746},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.454,"5min":1.454,"15min":1.454},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"min":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.222,"5min":1.222,"15min":1.222},"last":0.81699999999999995}]},{"osd":3,"last update":"Wed Sep 18 10:33:15 2024","interfaces":[{"interface":"back","average":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":1.5509999999999999,"5min":1.5509999999999999,"15min":1.5509999999999999},"last":1.036},{"interface":"front","average":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.6339999999999999,"5min":1.6339999999999999,"15min":1.6339999999999999},"last":0.80200000000000005}]},{"osd":4,"last update":"Wed Sep 18 10:33:15 2024","interfaces":[{"interface":"back","average":{"1min":1,"5min":1,"15min":1},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":1.6339999999999999,"5min":1.6339999999999999,"15min":1.6339999999999999},"last":0.98999999999999999},{"interface":"front","average":{"1min":1.093,"5min":1.093,"15min":1.093},"min":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.704,"5min":1.704,"15min":1.704},"last":1.135}]},{"osd":5,"last update":"Wed Sep 18 10:33:15 2024","interfaces":[{"interface":"back","average":{"1min":1.1699999999999999,"5min":1.1699999999999999,"15min":1.1699999999999999},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.9590000000000001,"5min":1.9590000000000001,"15min":1.9590000000000001},"last":1.1599999999999999},{"interface":"front","average":{"1min":1.2050000000000001,"5min":1.2050000000000001,"15min":1.2050000000000001},"min":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"max":{"1min":2.206,"5min":2.206,"15min":2.206},"last":1.0800000000000001}]},{"osd":6,"last update":"Wed Sep 18 10:33:15 2024","interfaces":[{"interface":"back","average":{"1min":1.157,"5min":1.157,"15min":1.157},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":1.74,"5min":1.74,"15min":1.74},"last":1.4159999999999999},{"interface":"front","average":{"1min":1.135,"5min":1.135,"15min":1.135},"min":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"max":{"1min":2.8140000000000001,"5min":2.8140000000000001,"15min":2.8140000000000001},"last":1.3919999999999999}]},{"osd":7,"last update":"Wed Sep 18 10:33:15 2024","interfaces":[{"interface":"back","average":{"1min":1.268,"5min":1.268,"15min":1.268},"min":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"max":{"1min":2.004,"5min":2.004,"15min":2.004},"last":2.6909999999999998},{"interface":"front","average":{"1min":1.357,"5min":1.357,"15min":1.357},"min":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"max":{"1min":2.7250000000000001,"5min":2.7250000000000001,"15min":2.7250000000000001},"last":1.8899999999999999}]},{"osd":8,"last update":"Wed Sep 18 10:33:15 2024","interfaces":[{"interface":"back","average":{"1min":1.3049999999999999,"5min":1.3049999999999999,"15min":1.3049999999999999},"min":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":2.1419999999999999,"5min":2.1419999999999999,"15min":2.1419999999999999},"last":2.5499999999999998},{"interface":"front","average":{"1min":1.3779999999999999,"5min":1.3779999999999999,"15min":1.3779999999999999},"min":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"max":{"1min":2.8980000000000001,"5min":2.8980000000000001,"15min":2.8980000000000001},"last":1.6279999999999999}]},{"osd":9,"last update":"Wed Sep 18 10:33:15 2024","interfaces":[{"interface":"back","average":{"1min":1.2989999999999999,"5min":1.2989999999999999,"15min":1.2989999999999999},"min":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"max":{"1min":2.3109999999999999,"5min":2.3109999999999999,"15min":2.3109999999999999},"last":1.8140000000000001},{"interface":"front","average":{"1min":1.333,"5min":1.333,"15min":1.333},"min":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":2.2639999999999998,"5min":2.2639999999999998,"15min":2.2639999999999998},"last":2.7719999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-18T10:34:03.631 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-18T10:34:03.922 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-18T10:34:03.925 INFO:teuthology.orchestra.run.smithi134.stderr:dumped all 2024-09-18T10:34:03.936 INFO:teuthology.orchestra.run.smithi134.stdout:{"pg_ready":true,"pg_map":{"version":105,"stamp":"2024-09-18T10:34:03.431290+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":12,"acting":12,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":10,"num_per_pool_osds":10,"num_per_pool_omap_osds":8,"kb":1048576000,"kb_used":2979912,"kb_used_data":6308,"kb_used_omap":16,"kb_used_meta":2973423,"kb_avail":1045596088,"statfs":{"total":1073741824000,"available":1070690394112,"internally_reserved":0,"allocated":6459392,"data_stored":3051408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":17040,"internal_metadata":3044785520},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000990"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":45,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-18T10:33:59.097752+0000","last_change":"2024-09-18T10:33:52.036028+0000","last_active":"2024-09-18T10:33:59.097752+0000","last_peered":"2024-09-18T10:33:59.097752+0000","last_clean":"2024-09-18T10:33:59.097752+0000","last_became_active":"2024-09-18T10:33:52.035591+0000","last_became_peered":"2024-09-18T10:33:52.035591+0000","last_unstale":"2024-09-18T10:33:59.097752+0000","last_undegraded":"2024-09-18T10:33:59.097752+0000","last_fullsized":"2024-09-18T10:33:59.097752+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T10:32:19.443095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:32:19.443095+0000","last_clean_scrub_stamp":"2024-09-18T10:32:19.443095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T18:06:23.838644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00038275899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":61,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-18T10:33:59.101190+0000","last_change":"2024-09-18T10:33:52.042125+0000","last_active":"2024-09-18T10:33:59.101190+0000","last_peered":"2024-09-18T10:33:59.101190+0000","last_clean":"2024-09-18T10:33:59.101190+0000","last_became_active":"2024-09-18T10:33:52.041895+0000","last_became_peered":"2024-09-18T10:33:52.041895+0000","last_unstale":"2024-09-18T10:33:59.101190+0000","last_undegraded":"2024-09-18T10:33:59.101190+0000","last_fullsized":"2024-09-18T10:33:59.101190+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T10:32:19.443095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:32:19.443095+0000","last_clean_scrub_stamp":"2024-09-18T10:32:19.443095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T15:18:02.048451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034975400000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":85,"reported_epoch":57,"state":"active+undersized","last_fresh":"2024-09-18T10:34:00.271592+0000","last_change":"2024-09-18T10:34:00.271514+0000","last_active":"2024-09-18T10:34:00.271592+0000","last_peered":"2024-09-18T10:34:00.271592+0000","last_clean":"2024-09-18T10:33:56.981057+0000","last_became_active":"2024-09-18T10:33:59.102191+0000","last_became_peered":"2024-09-18T10:33:59.102191+0000","last_unstale":"2024-09-18T10:34:00.271592+0000","last_undegraded":"2024-09-18T10:34:00.271592+0000","last_fullsized":"2024-09-18T10:33:59.101046+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T10:32:19.443095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:32:19.443095+0000","last_clean_scrub_stamp":"2024-09-18T10:32:19.443095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T13:05:55.380579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048480000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9],"acting":[9],"avail_no_missing":["9"],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":45,"reported_epoch":56,"state":"active+undersized+wait","last_fresh":"2024-09-18T10:33:59.102960+0000","last_change":"2024-09-18T10:33:59.102253+0000","last_active":"2024-09-18T10:33:59.102960+0000","last_peered":"2024-09-18T10:33:59.102960+0000","last_clean":"2024-09-18T10:33:47.951434+0000","last_became_active":"2024-09-18T10:33:59.102253+0000","last_became_peered":"2024-09-18T10:33:59.102253+0000","last_unstale":"2024-09-18T10:33:59.102960+0000","last_undegraded":"2024-09-18T10:33:59.102960+0000","last_fullsized":"2024-09-18T10:33:59.100957+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T10:32:19.443095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:32:19.443095+0000","last_clean_scrub_stamp":"2024-09-18T10:32:19.443095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T19:47:09.963588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6],"acting":[6],"avail_no_missing":["6"],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.3","version":"18'1","reported_seq":70,"reported_epoch":57,"state":"active+undersized+degraded+wait","last_fresh":"2024-09-18T10:34:00.106984+0000","last_change":"2024-09-18T10:34:00.106902+0000","last_active":"2024-09-18T10:34:00.106984+0000","last_peered":"2024-09-18T10:34:00.106984+0000","last_clean":"2024-09-18T10:33:59.061965+0000","last_became_active":"2024-09-18T10:34:00.106902+0000","last_became_peered":"2024-09-18T10:34:00.106902+0000","last_unstale":"2024-09-18T10:34:00.106984+0000","last_undegraded":"2024-09-18T10:34:00.106209+0000","last_fullsized":"2024-09-18T10:34:00.106121+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T10:32:19.443095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:32:19.443095+0000","last_clean_scrub_stamp":"2024-09-18T10:32:19.443095+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T21:39:17.714747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3],"acting":[3],"avail_no_missing":["3"],"object_location_counts":[{"shards":"3","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.2","version":"20'2","reported_seq":73,"reported_epoch":56,"state":"active+undersized+degraded+wait","last_fresh":"2024-09-18T10:33:59.102609+0000","last_change":"2024-09-18T10:33:59.102002+0000","last_active":"2024-09-18T10:33:59.102609+0000","last_peered":"2024-09-18T10:33:59.102609+0000","last_clean":"2024-09-18T10:33:58.228743+0000","last_became_active":"2024-09-18T10:33:59.102002+0000","last_became_peered":"2024-09-18T10:33:59.102002+0000","last_unstale":"2024-09-18T10:33:59.102609+0000","last_undegraded":"2024-09-18T10:33:59.100769+0000","last_fullsized":"2024-09-18T10:33:59.100508+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T10:32:19.443095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:32:19.443095+0000","last_clean_scrub_stamp":"2024-09-18T10:32:19.443095+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T19:07:49.365548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019475040000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4],"acting":[4],"avail_no_missing":["4"],"object_location_counts":[{"shards":"4","objects":1}],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":42,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-18T10:33:59.061961+0000","last_change":"2024-09-18T10:33:52.038532+0000","last_active":"2024-09-18T10:33:59.061961+0000","last_peered":"2024-09-18T10:33:59.061961+0000","last_clean":"2024-09-18T10:33:59.061961+0000","last_became_active":"2024-09-18T10:33:52.038239+0000","last_became_peered":"2024-09-18T10:33:52.038239+0000","last_unstale":"2024-09-18T10:33:59.061961+0000","last_undegraded":"2024-09-18T10:33:59.061961+0000","last_fullsized":"2024-09-18T10:33:59.061961+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T10:32:19.443095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:32:19.443095+0000","last_clean_scrub_stamp":"2024-09-18T10:32:19.443095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T18:34:12.717302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062722600000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":62,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-18T10:33:59.400931+0000","last_change":"2024-09-18T10:33:52.037375+0000","last_active":"2024-09-18T10:33:59.400931+0000","last_peered":"2024-09-18T10:33:59.400931+0000","last_clean":"2024-09-18T10:33:59.400931+0000","last_became_active":"2024-09-18T10:33:52.037028+0000","last_became_peered":"2024-09-18T10:33:52.037028+0000","last_unstale":"2024-09-18T10:33:59.400931+0000","last_undegraded":"2024-09-18T10:33:59.400931+0000","last_fullsized":"2024-09-18T10:33:59.400931+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T10:32:19.443095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:32:19.443095+0000","last_clean_scrub_stamp":"2024-09-18T10:32:19.443095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T19:41:50.364436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9],"acting":[5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":12,"acting":12,"num_store_stats":9}],"osd_stats":[{"osd":9,"up_from":41,"seq":176093659159,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":298004,"kb_used_data":648,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":104559596,"statfs":{"total":107374182400,"available":107069026304,"internally_reserved":0,"allocated":663552,"data_stored":316651,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[2,3,4,5,6,7,8],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 10:33:18 2024","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":1.1930000000000001,"5min":1.1930000000000001,"15min":1.1930000000000001},"last":1.121},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.3029999999999999,"5min":1.3029999999999999,"15min":1.3029999999999999},"last":0.64900000000000002}]},{"osd":1,"last update":"Wed Sep 18 10:33:18 2024","interfaces":[{"interface":"back","average":{"1min":0.873,"5min":0.873,"15min":0.873},"min":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.3919999999999999,"5min":1.3919999999999999,"15min":1.3919999999999999},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":2.4359999999999999,"5min":2.4359999999999999,"15min":2.4359999999999999},"last":1.6379999999999999}]},{"osd":2,"last update":"Wed Sep 18 10:33:18 2024","interfaces":[{"interface":"back","average":{"1min":1.03,"5min":1.03,"15min":1.03},"min":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":1.526,"5min":1.526,"15min":1.526},"last":0.45300000000000001},{"interface":"front","average":{"1min":1.044,"5min":1.044,"15min":1.044},"min":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"max":{"1min":1.591,"5min":1.591,"15min":1.591},"last":0.52000000000000002}]},{"osd":3,"last update":"Wed Sep 18 10:33:18 2024","interfaces":[{"interface":"back","average":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.542,"5min":1.542,"15min":1.542},"last":0.63800000000000001},{"interface":"front","average":{"1min":1.274,"5min":1.274,"15min":1.274},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.8440000000000001,"5min":1.8440000000000001,"15min":1.8440000000000001},"last":0.98099999999999998}]},{"osd":4,"last update":"Wed Sep 18 10:33:18 2024","interfaces":[{"interface":"back","average":{"1min":1.2430000000000001,"5min":1.2430000000000001,"15min":1.2430000000000001},"min":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"max":{"1min":1.8180000000000001,"5min":1.8180000000000001,"15min":1.8180000000000001},"last":0.67300000000000004},{"interface":"front","average":{"1min":1.276,"5min":1.276,"15min":1.276},"min":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"max":{"1min":1.681,"5min":1.681,"15min":1.681},"last":0.96399999999999997}]},{"osd":5,"last update":"Wed Sep 18 10:33:18 2024","interfaces":[{"interface":"back","average":{"1min":1.3140000000000001,"5min":1.3140000000000001,"15min":1.3140000000000001},"min":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"max":{"1min":1.8600000000000001,"5min":1.8600000000000001,"15min":1.8600000000000001},"last":1.0269999999999999},{"interface":"front","average":{"1min":1.3009999999999999,"5min":1.3009999999999999,"15min":1.3009999999999999},"min":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"max":{"1min":1.7090000000000001,"5min":1.7090000000000001,"15min":1.7090000000000001},"last":0.78100000000000003}]},{"osd":6,"last update":"Wed Sep 18 10:33:18 2024","interfaces":[{"interface":"back","average":{"1min":1.3029999999999999,"5min":1.3029999999999999,"15min":1.3029999999999999},"min":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":2.778,"5min":2.778,"15min":2.778},"last":0.86599999999999999},{"interface":"front","average":{"1min":1.345,"5min":1.345,"15min":1.345},"min":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":1.921,"5min":1.921,"15min":1.921},"last":1.075}]},{"osd":7,"last update":"Wed Sep 18 10:33:18 2024","interfaces":[{"interface":"back","average":{"1min":1.3340000000000001,"5min":1.3340000000000001,"15min":1.3340000000000001},"min":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"max":{"1min":1.6830000000000001,"5min":1.6830000000000001,"15min":1.6830000000000001},"last":1.1439999999999999},{"interface":"front","average":{"1min":1.417,"5min":1.417,"15min":1.417},"min":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"max":{"1min":3.0390000000000001,"5min":3.0390000000000001,"15min":3.0390000000000001},"last":0.90600000000000003}]},{"osd":8,"last update":"Wed Sep 18 10:33:18 2024","interfaces":[{"interface":"back","average":{"1min":1.4330000000000001,"5min":1.4330000000000001,"15min":1.4330000000000001},"min":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"max":{"1min":2.8500000000000001,"5min":2.8500000000000001,"15min":2.8500000000000001},"last":0.93999999999999995},{"interface":"front","average":{"1min":1.4079999999999999,"5min":1.4079999999999999,"15min":1.4079999999999999},"min":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"max":{"1min":2.0070000000000001,"5min":2.0070000000000001,"15min":2.0070000000000001},"last":1.1910000000000001}]}]},{"osd":8,"up_from":48,"seq":206158430232,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":104857600,"kb_used":297956,"kb_used_data":600,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":104559644,"statfs":{"total":107374182400,"available":107069075456,"internally_reserved":0,"allocated":614400,"data_stored":285923,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,6,7,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 10:33:16 2024","interfaces":[{"interface":"back","average":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":1.26,"5min":1.26,"15min":1.26},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":1.238,"5min":1.238,"15min":1.238},"last":0.86299999999999999}]},{"osd":1,"last update":"Wed Sep 18 10:33:16 2024","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.751,"15min":0.751},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":1.129,"5min":1.129,"15min":1.129},"last":1.1359999999999999},{"interface":"front","average":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"min":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.2390000000000001,"5min":1.2390000000000001,"15min":1.2390000000000001},"last":0.92600000000000005}]},{"osd":2,"last update":"Wed Sep 18 10:33:16 2024","interfaces":[{"interface":"back","average":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":1.6439999999999999,"5min":1.6439999999999999,"15min":1.6439999999999999},"last":1.238},{"interface":"front","average":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":1.2529999999999999,"5min":1.2529999999999999,"15min":1.2529999999999999},"last":0.82299999999999995}]},{"osd":3,"last update":"Wed Sep 18 10:33:16 2024","interfaces":[{"interface":"back","average":{"1min":1.109,"5min":1.109,"15min":1.109},"min":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"max":{"1min":1.538,"5min":1.538,"15min":1.538},"last":1.458},{"interface":"front","average":{"1min":1.0389999999999999,"5min":1.0389999999999999,"15min":1.0389999999999999},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.4810000000000001,"5min":1.4810000000000001,"15min":1.4810000000000001},"last":1.581}]},{"osd":4,"last update":"Wed Sep 18 10:33:16 2024","interfaces":[{"interface":"back","average":{"1min":1.0620000000000001,"5min":1.0620000000000001,"15min":1.0620000000000001},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.8169999999999999,"5min":1.8169999999999999,"15min":1.8169999999999999},"last":1.917},{"interface":"front","average":{"1min":1.2549999999999999,"5min":1.2549999999999999,"15min":1.2549999999999999},"min":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"max":{"1min":2.1480000000000001,"5min":2.1480000000000001,"15min":2.1480000000000001},"last":1.3600000000000001}]},{"osd":5,"last update":"Wed Sep 18 10:33:16 2024","interfaces":[{"interface":"back","average":{"1min":1.216,"5min":1.216,"15min":1.216},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":2.3109999999999999,"5min":2.3109999999999999,"15min":2.3109999999999999},"last":1.637},{"interface":"front","average":{"1min":1.161,"5min":1.161,"15min":1.161},"min":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":2.0169999999999999,"5min":2.0169999999999999,"15min":2.0169999999999999},"last":1.7110000000000001}]},{"osd":6,"last update":"Wed Sep 18 10:33:16 2024","interfaces":[{"interface":"back","average":{"1min":1.2889999999999999,"5min":1.2889999999999999,"15min":1.2889999999999999},"min":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"max":{"1min":2.1899999999999999,"5min":2.1899999999999999,"15min":2.1899999999999999},"last":1.9470000000000001},{"interface":"front","average":{"1min":1.1799999999999999,"5min":1.1799999999999999,"15min":1.1799999999999999},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":2.069,"5min":2.069,"15min":2.069},"last":1.764}]},{"osd":7,"last update":"Wed Sep 18 10:33:16 2024","interfaces":[{"interface":"back","average":{"1min":1.2589999999999999,"5min":1.2589999999999999,"15min":1.2589999999999999},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":2.2130000000000001,"5min":2.2130000000000001,"15min":2.2130000000000001},"last":1.5129999999999999},{"interface":"front","average":{"1min":1.4039999999999999,"5min":1.4039999999999999,"15min":1.4039999999999999},"min":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"max":{"1min":2.2450000000000001,"5min":2.2450000000000001,"15min":2.2450000000000001},"last":2.0459999999999998}]},{"osd":9,"last update":"Wed Sep 18 10:33:16 2024","interfaces":[{"interface":"back","average":{"1min":1.304,"5min":1.304,"15min":1.304},"min":{"1min":0.63,"5min":0.63,"15min":0.63},"max":{"1min":2.1800000000000002,"5min":2.1800000000000002,"15min":2.1800000000000002},"last":2.0790000000000002},{"interface":"front","average":{"1min":1.3440000000000001,"5min":1.3440000000000001,"15min":1.3440000000000001},"min":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"max":{"1min":2.3969999999999998,"5min":2.3969999999999998,"15min":2.3969999999999998},"last":1.875}]}]},{"osd":7,"up_from":48,"seq":206158430231,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":297968,"kb_used_data":612,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":104559632,"statfs":{"total":107374182400,"available":107069063168,"internally_reserved":0,"allocated":626688,"data_stored":293647,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,6,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 10:33:17 2024","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.202,"5min":1.202,"15min":1.202},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.75,"5min":0.75,"15min":0.75},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":1.486,"5min":1.486,"15min":1.486},"last":0.98799999999999999}]},{"osd":1,"last update":"Wed Sep 18 10:33:17 2024","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.3009999999999999,"5min":1.3009999999999999,"15min":1.3009999999999999},"last":1.085},{"interface":"front","average":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.8480000000000001,"5min":1.8480000000000001,"15min":1.8480000000000001},"last":0.73799999999999999}]},{"osd":2,"last update":"Wed Sep 18 10:33:17 2024","interfaces":[{"interface":"back","average":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":1.5980000000000001,"5min":1.5980000000000001,"15min":1.5980000000000001},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.3440000000000001,"5min":1.3440000000000001,"15min":1.3440000000000001},"last":0.77800000000000002}]},{"osd":3,"last update":"Wed Sep 18 10:33:17 2024","interfaces":[{"interface":"back","average":{"1min":1.1000000000000001,"5min":1.1000000000000001,"15min":1.1000000000000001},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.9319999999999999,"5min":1.9319999999999999,"15min":1.9319999999999999},"last":0.85099999999999998},{"interface":"front","average":{"1min":1.0720000000000001,"5min":1.0720000000000001,"15min":1.0720000000000001},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.8879999999999999,"5min":1.8879999999999999,"15min":1.8879999999999999},"last":0.91400000000000003}]},{"osd":4,"last update":"Wed Sep 18 10:33:17 2024","interfaces":[{"interface":"back","average":{"1min":1.097,"5min":1.097,"15min":1.097},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":2.1379999999999999,"5min":2.1379999999999999,"15min":2.1379999999999999},"last":1.8680000000000001},{"interface":"front","average":{"1min":1.1060000000000001,"5min":1.1060000000000001,"15min":1.1060000000000001},"min":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":2.0110000000000001,"5min":2.0110000000000001,"15min":2.0110000000000001},"last":0.90400000000000003}]},{"osd":5,"last update":"Wed Sep 18 10:33:17 2024","interfaces":[{"interface":"back","average":{"1min":1.194,"5min":1.194,"15min":1.194},"min":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"max":{"1min":2.4889999999999999,"5min":2.4889999999999999,"15min":2.4889999999999999},"last":1.0329999999999999},{"interface":"front","average":{"1min":1.2230000000000001,"5min":1.2230000000000001,"15min":1.2230000000000001},"min":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"max":{"1min":2.4289999999999998,"5min":2.4289999999999998,"15min":2.4289999999999998},"last":2.056}]},{"osd":6,"last update":"Wed Sep 18 10:33:17 2024","interfaces":[{"interface":"back","average":{"1min":1.2769999999999999,"5min":1.2769999999999999,"15min":1.2769999999999999},"min":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"max":{"1min":2.0950000000000002,"5min":2.0950000000000002,"15min":2.0950000000000002},"last":1.341},{"interface":"front","average":{"1min":1.218,"5min":1.218,"15min":1.218},"min":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":1.9450000000000001,"5min":1.9450000000000001,"15min":1.9450000000000001},"last":1.1930000000000001}]},{"osd":8,"last update":"Wed Sep 18 10:33:17 2024","interfaces":[{"interface":"back","average":{"1min":1.2969999999999999,"5min":1.2969999999999999,"15min":1.2969999999999999},"min":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"max":{"1min":2.6389999999999998,"5min":2.6389999999999998,"15min":2.6389999999999998},"last":1.9810000000000001},{"interface":"front","average":{"1min":1.3600000000000001,"5min":1.3600000000000001,"15min":1.3600000000000001},"min":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"max":{"1min":2.1520000000000001,"5min":2.1520000000000001,"15min":2.1520000000000001},"last":1.2789999999999999}]},{"osd":9,"last update":"Wed Sep 18 10:33:17 2024","interfaces":[{"interface":"back","average":{"1min":1.2470000000000001,"5min":1.2470000000000001,"15min":1.2470000000000001},"min":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":2.0680000000000001,"5min":2.0680000000000001,"15min":2.0680000000000001},"last":2.2469999999999999},{"interface":"front","average":{"1min":1.282,"5min":1.282,"15min":1.282},"min":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"max":{"1min":1.8460000000000001,"5min":1.8460000000000001,"15min":1.8460000000000001},"last":2.9929999999999999}]}]},{"osd":6,"up_from":48,"seq":206158430231,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":297988,"kb_used_data":624,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":104559612,"statfs":{"total":107374182400,"available":107069042688,"internally_reserved":0,"allocated":638976,"data_stored":301295,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,7,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 10:33:17 2024","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":1.073,"5min":1.073,"15min":1.073},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":1.3720000000000001,"5min":1.3720000000000001,"15min":1.3720000000000001},"last":0.70499999999999996}]},{"osd":1,"last update":"Wed Sep 18 10:33:17 2024","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.751,"15min":0.751},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":1.6599999999999999,"5min":1.6599999999999999,"15min":1.6599999999999999},"last":1.095},{"interface":"front","average":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":2.0459999999999998,"5min":2.0459999999999998,"15min":2.0459999999999998},"last":0.61399999999999999}]},{"osd":2,"last update":"Wed Sep 18 10:33:17 2024","interfaces":[{"interface":"back","average":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":1.1399999999999999,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":0.77000000000000002},{"interface":"front","average":{"1min":1.052,"5min":1.052,"15min":1.052},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":1.907,"5min":1.907,"15min":1.907},"last":0.92400000000000004}]},{"osd":3,"last update":"Wed Sep 18 10:33:17 2024","interfaces":[{"interface":"back","average":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.7490000000000001,"5min":1.7490000000000001,"15min":1.7490000000000001},"last":0.749},{"interface":"front","average":{"1min":1.1830000000000001,"5min":1.1830000000000001,"15min":1.1830000000000001},"min":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"max":{"1min":1.8680000000000001,"5min":1.8680000000000001,"15min":1.8680000000000001},"last":1.1679999999999999}]},{"osd":4,"last update":"Wed Sep 18 10:33:17 2024","interfaces":[{"interface":"back","average":{"1min":1.2070000000000001,"5min":1.2070000000000001,"15min":1.2070000000000001},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.976,"5min":1.976,"15min":1.976},"last":1.25},{"interface":"front","average":{"1min":1.127,"5min":1.127,"15min":1.127},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":2.544,"5min":2.544,"15min":2.544},"last":1.6200000000000001}]},{"osd":5,"last update":"Wed Sep 18 10:33:17 2024","interfaces":[{"interface":"back","average":{"1min":1.2170000000000001,"5min":1.2170000000000001,"15min":1.2170000000000001},"min":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"max":{"1min":2.1539999999999999,"5min":2.1539999999999999,"15min":2.1539999999999999},"last":1.698},{"interface":"front","average":{"1min":1.296,"5min":1.296,"15min":1.296},"min":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":2.2469999999999999,"5min":2.2469999999999999,"15min":2.2469999999999999},"last":1.5649999999999999}]},{"osd":7,"last update":"Wed Sep 18 10:33:17 2024","interfaces":[{"interface":"back","average":{"1min":1.246,"5min":1.246,"15min":1.246},"min":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":2.6440000000000001,"5min":2.6440000000000001,"15min":2.6440000000000001},"last":1.2769999999999999},{"interface":"front","average":{"1min":1.7609999999999999,"5min":1.7609999999999999,"15min":1.7609999999999999},"min":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"max":{"1min":10.385,"5min":10.385,"15min":10.385},"last":1.633}]},{"osd":8,"last update":"Wed Sep 18 10:33:17 2024","interfaces":[{"interface":"back","average":{"1min":1.3640000000000001,"5min":1.3640000000000001,"15min":1.3640000000000001},"min":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"max":{"1min":2.444,"5min":2.444,"15min":2.444},"last":1.516},{"interface":"front","average":{"1min":1.274,"5min":1.274,"15min":1.274},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":2.714,"5min":2.714,"15min":2.714},"last":1.3580000000000001}]},{"osd":9,"last update":"Wed Sep 18 10:33:17 2024","interfaces":[{"interface":"back","average":{"1min":1.2789999999999999,"5min":1.2789999999999999,"15min":1.2789999999999999},"min":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"max":{"1min":2.2879999999999998,"5min":2.2879999999999998,"15min":2.2879999999999998},"last":1.768},{"interface":"front","average":{"1min":1.3240000000000001,"5min":1.3240000000000001,"15min":1.3240000000000001},"min":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":2.5960000000000001,"5min":2.5960000000000001,"15min":2.5960000000000001},"last":1.669}]}]},{"osd":5,"up_from":48,"seq":206158430233,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":298000,"kb_used_data":636,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":104559600,"statfs":{"total":107374182400,"available":107069030400,"internally_reserved":0,"allocated":651264,"data_stored":308977,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[2,3,4,6,7,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 10:33:18 2024","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.111,"5min":1.111,"15min":1.111},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"min":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.5249999999999999,"5min":1.5249999999999999,"15min":1.5249999999999999},"last":0.63}]},{"osd":1,"last update":"Wed Sep 18 10:33:18 2024","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.173,"5min":1.173,"15min":1.173},"last":0.79600000000000004},{"interface":"front","average":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.8280000000000001,"5min":1.8280000000000001,"15min":1.8280000000000001},"last":0.88500000000000001}]},{"osd":2,"last update":"Wed Sep 18 10:33:18 2024","interfaces":[{"interface":"back","average":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"min":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":1.4159999999999999,"5min":1.4159999999999999,"15min":1.4159999999999999},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"min":{"1min":0.627,"5min":0.627,"15min":0.627},"max":{"1min":2.3849999999999998,"5min":2.3849999999999998,"15min":2.3849999999999998},"last":0.63200000000000001}]},{"osd":3,"last update":"Wed Sep 18 10:33:18 2024","interfaces":[{"interface":"back","average":{"1min":1.079,"5min":1.079,"15min":1.079},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.923,"5min":1.923,"15min":1.923},"last":0.57099999999999995},{"interface":"front","average":{"1min":1.268,"5min":1.268,"15min":1.268},"min":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"max":{"1min":2.073,"5min":2.073,"15min":2.073},"last":0.72799999999999998}]},{"osd":4,"last update":"Wed Sep 18 10:33:18 2024","interfaces":[{"interface":"back","average":{"1min":1.123,"5min":1.123,"15min":1.123},"min":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"max":{"1min":2.141,"5min":2.141,"15min":2.141},"last":0.60699999999999998},{"interface":"front","average":{"1min":1.2490000000000001,"5min":1.2490000000000001,"15min":1.2490000000000001},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":2.3300000000000001,"5min":2.3300000000000001,"15min":2.3300000000000001},"last":1.05}]},{"osd":6,"last update":"Wed Sep 18 10:33:18 2024","interfaces":[{"interface":"back","average":{"1min":1.2769999999999999,"5min":1.2769999999999999,"15min":1.2769999999999999},"min":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"max":{"1min":1.806,"5min":1.806,"15min":1.806},"last":0.81499999999999995},{"interface":"front","average":{"1min":1.274,"5min":1.274,"15min":1.274},"min":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"max":{"1min":2.6120000000000001,"5min":2.6120000000000001,"15min":2.6120000000000001},"last":0.70199999999999996}]},{"osd":7,"last update":"Wed Sep 18 10:33:18 2024","interfaces":[{"interface":"back","average":{"1min":1.3380000000000001,"5min":1.3380000000000001,"15min":1.3380000000000001},"min":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"max":{"1min":2.661,"5min":2.661,"15min":2.661},"last":1.018},{"interface":"front","average":{"1min":1.3819999999999999,"5min":1.3819999999999999,"15min":1.3819999999999999},"min":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"max":{"1min":2.548,"5min":2.548,"15min":2.548},"last":0.89600000000000002}]},{"osd":8,"last update":"Wed Sep 18 10:33:18 2024","interfaces":[{"interface":"back","average":{"1min":1.359,"5min":1.359,"15min":1.359},"min":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"max":{"1min":2.7080000000000002,"5min":2.7080000000000002,"15min":2.7080000000000002},"last":0.96599999999999997},{"interface":"front","average":{"1min":1.327,"5min":1.327,"15min":1.327},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":2.4399999999999999,"5min":2.4399999999999999,"15min":2.4399999999999999},"last":1.169}]},{"osd":9,"last update":"Wed Sep 18 10:33:18 2024","interfaces":[{"interface":"back","average":{"1min":1.3859999999999999,"5min":1.3859999999999999,"15min":1.3859999999999999},"min":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"max":{"1min":2.2130000000000001,"5min":2.2130000000000001,"15min":2.2130000000000001},"last":0.92700000000000005},{"interface":"front","average":{"1min":1.4019999999999999,"5min":1.4019999999999999,"15min":1.4019999999999999},"min":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"max":{"1min":2.782,"5min":2.782,"15min":2.782},"last":1.0860000000000001}]}]},{"osd":4,"up_from":48,"seq":206158430231,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":297992,"kb_used_data":628,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":104559608,"statfs":{"total":107374182400,"available":107069038592,"internally_reserved":0,"allocated":643072,"data_stored":301314,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,5,6,7,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 10:33:16 2024","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":1.0960000000000001,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.745,"5min":0.745,"15min":0.745},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.171,"5min":1.171,"15min":1.171},"last":0.70799999999999996}]},{"osd":1,"last update":"Wed Sep 18 10:33:16 2024","interfaces":[{"interface":"back","average":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":1.1579999999999999,"5min":1.1579999999999999,"15min":1.1579999999999999},"last":1.0820000000000001},{"interface":"front","average":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.583,"5min":1.583,"15min":1.583},"last":0.77400000000000002}]},{"osd":2,"last update":"Wed Sep 18 10:33:16 2024","interfaces":[{"interface":"back","average":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":2.1139999999999999,"5min":2.1139999999999999,"15min":2.1139999999999999},"last":1.165},{"interface":"front","average":{"1min":0.995,"5min":0.995,"15min":0.995},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.5660000000000001,"5min":1.5660000000000001,"15min":1.5660000000000001},"last":0.56100000000000005}]},{"osd":3,"last update":"Wed Sep 18 10:33:16 2024","interfaces":[{"interface":"back","average":{"1min":1.0620000000000001,"5min":1.0620000000000001,"15min":1.0620000000000001},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.6200000000000001,"5min":1.6200000000000001,"15min":1.6200000000000001},"last":0.91400000000000003},{"interface":"front","average":{"1min":1.1699999999999999,"5min":1.1699999999999999,"15min":1.1699999999999999},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.732,"5min":1.732,"15min":1.732},"last":0.876}]},{"osd":5,"last update":"Wed Sep 18 10:33:16 2024","interfaces":[{"interface":"back","average":{"1min":1.159,"5min":1.159,"15min":1.159},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":2.2509999999999999,"5min":2.2509999999999999,"15min":2.2509999999999999},"last":1.0489999999999999},{"interface":"front","average":{"1min":1.1830000000000001,"5min":1.1830000000000001,"15min":1.1830000000000001},"min":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":1.873,"5min":1.873,"15min":1.873},"last":1.0049999999999999}]},{"osd":6,"last update":"Wed Sep 18 10:33:16 2024","interfaces":[{"interface":"back","average":{"1min":1.214,"5min":1.214,"15min":1.214},"min":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"max":{"1min":2.2879999999999998,"5min":2.2879999999999998,"15min":2.2879999999999998},"last":1.232},{"interface":"front","average":{"1min":1.25,"5min":1.25,"15min":1.25},"min":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":2.661,"5min":2.661,"15min":2.661},"last":1.137}]},{"osd":7,"last update":"Wed Sep 18 10:33:16 2024","interfaces":[{"interface":"back","average":{"1min":1.319,"5min":1.319,"15min":1.319},"min":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"max":{"1min":1.895,"5min":1.895,"15min":1.895},"last":1.415},{"interface":"front","average":{"1min":1.2450000000000001,"5min":1.2450000000000001,"15min":1.2450000000000001},"min":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":2.0579999999999998,"5min":2.0579999999999998,"15min":2.0579999999999998},"last":1.6619999999999999}]},{"osd":8,"last update":"Wed Sep 18 10:33:16 2024","interfaces":[{"interface":"back","average":{"1min":1.278,"5min":1.278,"15min":1.278},"min":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":2.173,"5min":2.173,"15min":2.173},"last":1.573},{"interface":"front","average":{"1min":1.3240000000000001,"5min":1.3240000000000001,"15min":1.3240000000000001},"min":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":1.9750000000000001,"5min":1.9750000000000001,"15min":1.9750000000000001},"last":1.375}]},{"osd":9,"last update":"Wed Sep 18 10:33:16 2024","interfaces":[{"interface":"back","average":{"1min":1.327,"5min":1.327,"15min":1.327},"min":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":2.448,"5min":2.448,"15min":2.448},"last":1.4419999999999999},{"interface":"front","average":{"1min":1.389,"5min":1.389,"15min":1.389},"min":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"max":{"1min":2.242,"5min":2.242,"15min":2.242},"last":1.474}]}]},{"osd":3,"up_from":48,"seq":206158430233,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":298000,"kb_used_data":636,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":104559600,"statfs":{"total":107374182400,"available":107069030400,"internally_reserved":0,"allocated":651264,"data_stored":308977,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[2,4,5,6,7,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 10:33:15 2024","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.095,"5min":1.095,"15min":1.095},"last":1.4199999999999999},{"interface":"front","average":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"min":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":2.121,"5min":2.121,"15min":2.121},"last":0.84499999999999997}]},{"osd":1,"last update":"Wed Sep 18 10:33:15 2024","interfaces":[{"interface":"back","average":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":1.9259999999999999,"5min":1.9259999999999999,"15min":1.9259999999999999},"last":1.1140000000000001},{"interface":"front","average":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"min":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":1.2090000000000001,"5min":1.2090000000000001,"15min":1.2090000000000001},"last":0.80800000000000005}]},{"osd":2,"last update":"Wed Sep 18 10:33:15 2024","interfaces":[{"interface":"back","average":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.204,"5min":1.204,"15min":1.204},"last":0.90900000000000003},{"interface":"front","average":{"1min":1.1870000000000001,"5min":1.1870000000000001,"15min":1.1870000000000001},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":3.2040000000000002,"5min":3.2040000000000002,"15min":3.2040000000000002},"last":2.6230000000000002}]},{"osd":4,"last update":"Wed Sep 18 10:33:15 2024","interfaces":[{"interface":"back","average":{"1min":1.071,"5min":1.071,"15min":1.071},"min":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":1.49,"5min":1.49,"15min":1.49},"last":0.82799999999999996},{"interface":"front","average":{"1min":1.266,"5min":1.266,"15min":1.266},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":2.7519999999999998,"5min":2.7519999999999998,"15min":2.7519999999999998},"last":1.016}]},{"osd":5,"last update":"Wed Sep 18 10:33:15 2024","interfaces":[{"interface":"back","average":{"1min":1.1830000000000001,"5min":1.1830000000000001,"15min":1.1830000000000001},"min":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"max":{"1min":2.0630000000000002,"5min":2.0630000000000002,"15min":2.0630000000000002},"last":0.71299999999999997},{"interface":"front","average":{"1min":1.1259999999999999,"5min":1.1259999999999999,"15min":1.1259999999999999},"min":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"max":{"1min":1.5149999999999999,"5min":1.5149999999999999,"15min":1.5149999999999999},"last":1.115}]},{"osd":6,"last update":"Wed Sep 18 10:33:15 2024","interfaces":[{"interface":"back","average":{"1min":1.3620000000000001,"5min":1.3620000000000001,"15min":1.3620000000000001},"min":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":4.1029999999999998,"5min":4.1029999999999998,"15min":4.1029999999999998},"last":0.97699999999999998},{"interface":"front","average":{"1min":1.268,"5min":1.268,"15min":1.268},"min":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"max":{"1min":2.2789999999999999,"5min":2.2789999999999999,"15min":2.2789999999999999},"last":1.427}]},{"osd":7,"last update":"Wed Sep 18 10:33:15 2024","interfaces":[{"interface":"back","average":{"1min":1.2010000000000001,"5min":1.2010000000000001,"15min":1.2010000000000001},"min":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"max":{"1min":2.1880000000000002,"5min":2.1880000000000002,"15min":2.1880000000000002},"last":2.7589999999999999},{"interface":"front","average":{"1min":1.282,"5min":1.282,"15min":1.282},"min":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"max":{"1min":3.0409999999999999,"5min":3.0409999999999999,"15min":3.0409999999999999},"last":1.2}]},{"osd":8,"last update":"Wed Sep 18 10:33:15 2024","interfaces":[{"interface":"back","average":{"1min":1.5109999999999999,"5min":1.5109999999999999,"15min":1.5109999999999999},"min":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"max":{"1min":3.5590000000000002,"5min":3.5590000000000002,"15min":3.5590000000000002},"last":1.3360000000000001},{"interface":"front","average":{"1min":1.3540000000000001,"5min":1.3540000000000001,"15min":1.3540000000000001},"min":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"max":{"1min":2.3919999999999999,"5min":2.3919999999999999,"15min":2.3919999999999999},"last":1.4850000000000001}]},{"osd":9,"last update":"Wed Sep 18 10:33:15 2024","interfaces":[{"interface":"back","average":{"1min":1.379,"5min":1.379,"15min":1.379},"min":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"max":{"1min":3.1259999999999999,"5min":3.1259999999999999,"15min":3.1259999999999999},"last":1.522},{"interface":"front","average":{"1min":1.5109999999999999,"5min":1.5109999999999999,"15min":1.5109999999999999},"min":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"max":{"1min":3.6339999999999999,"5min":3.6339999999999999,"15min":3.6339999999999999},"last":1.6020000000000001}]}]},{"osd":2,"up_from":48,"seq":206158430231,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":104857600,"kb_used":297996,"kb_used_data":636,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":104559604,"statfs":{"total":107374182400,"available":107069034496,"internally_reserved":0,"allocated":651264,"data_stored":308977,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,3,4,5,6,7,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 10:33:18 2024","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1.109,"5min":1.109,"15min":1.109},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":1.5029999999999999,"5min":1.5029999999999999,"15min":1.5029999999999999},"last":0.69299999999999995}]},{"osd":1,"last update":"Wed Sep 18 10:33:18 2024","interfaces":[{"interface":"back","average":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"min":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.651,"5min":1.651,"15min":1.651},"last":0.71799999999999997},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":1.1559999999999999,"5min":1.1559999999999999,"15min":1.1559999999999999},"last":1.1599999999999999}]},{"osd":3,"last update":"Wed Sep 18 10:33:18 2024","interfaces":[{"interface":"back","average":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":1.2869999999999999,"5min":1.2869999999999999,"15min":1.2869999999999999},"last":0.92800000000000005},{"interface":"front","average":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.7390000000000001,"5min":1.7390000000000001,"15min":1.7390000000000001},"last":0.83899999999999997}]},{"osd":4,"last update":"Wed Sep 18 10:33:18 2024","interfaces":[{"interface":"back","average":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"min":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":1.333,"5min":1.333,"15min":1.333},"last":1.329},{"interface":"front","average":{"1min":1.151,"5min":1.151,"15min":1.151},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.7869999999999999,"5min":1.7869999999999999,"15min":1.7869999999999999},"last":0.89300000000000002}]},{"osd":5,"last update":"Wed Sep 18 10:33:18 2024","interfaces":[{"interface":"back","average":{"1min":1.1100000000000001,"5min":1.1100000000000001,"15min":1.1100000000000001},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.597,"5min":1.597,"15min":1.597},"last":1.2909999999999999},{"interface":"front","average":{"1min":1.246,"5min":1.246,"15min":1.246},"min":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"max":{"1min":1.873,"5min":1.873,"15min":1.873},"last":1.0029999999999999}]},{"osd":6,"last update":"Wed Sep 18 10:33:18 2024","interfaces":[{"interface":"back","average":{"1min":1.0649999999999999,"5min":1.0649999999999999,"15min":1.0649999999999999},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":1.696,"5min":1.696,"15min":1.696},"last":1.6259999999999999},{"interface":"front","average":{"1min":1.1479999999999999,"5min":1.1479999999999999,"15min":1.1479999999999999},"min":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"max":{"1min":1.5289999999999999,"5min":1.5289999999999999,"15min":1.5289999999999999},"last":1.605}]},{"osd":7,"last update":"Wed Sep 18 10:33:18 2024","interfaces":[{"interface":"back","average":{"1min":1.268,"5min":1.268,"15min":1.268},"min":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":1.9350000000000001,"5min":1.9350000000000001,"15min":1.9350000000000001},"last":1.492},{"interface":"front","average":{"1min":1.244,"5min":1.244,"15min":1.244},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":2.6419999999999999,"5min":2.6419999999999999,"15min":2.6419999999999999},"last":1.534}]},{"osd":8,"last update":"Wed Sep 18 10:33:18 2024","interfaces":[{"interface":"back","average":{"1min":1.306,"5min":1.306,"15min":1.306},"min":{"1min":1,"5min":1,"15min":1},"max":{"1min":1.613,"5min":1.613,"15min":1.613},"last":1.093},{"interface":"front","average":{"1min":1.278,"5min":1.278,"15min":1.278},"min":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":1.9830000000000001,"5min":1.9830000000000001,"15min":1.9830000000000001},"last":1.7}]},{"osd":9,"last update":"Wed Sep 18 10:33:18 2024","interfaces":[{"interface":"back","average":{"1min":1.2210000000000001,"5min":1.2210000000000001,"15min":1.2210000000000001},"min":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"max":{"1min":2.4220000000000002,"5min":2.4220000000000002,"15min":2.4220000000000002},"last":1.5720000000000001},{"interface":"front","average":{"1min":1.337,"5min":1.337,"15min":1.337},"min":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"max":{"1min":2.1349999999999998,"5min":2.1349999999999998,"15min":2.1349999999999998},"last":1.0609999999999999}]}]},{"osd":1,"up_from":0,"seq":24,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":298008,"kb_used_data":652,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":104559592,"statfs":{"total":107374182400,"available":107069022208,"internally_reserved":0,"allocated":667648,"data_stored":316670,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[2,3,4,5,6,7,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 10:33:17 2024","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":1.0980000000000001,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.89900000000000002},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"min":{"1min":0.248,"5min":0.248,"15min":0.248},"max":{"1min":1.3,"5min":1.3,"15min":1.3},"last":0.58699999999999997}]},{"osd":2,"last update":"Wed Sep 18 10:33:17 2024","interfaces":[{"interface":"back","average":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.206,"5min":1.206,"15min":1.206},"last":1.1020000000000001},{"interface":"front","average":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":1.54,"5min":1.54,"15min":1.54},"last":0.83199999999999996}]},{"osd":3,"last update":"Wed Sep 18 10:33:17 2024","interfaces":[{"interface":"back","average":{"1min":1.129,"5min":1.129,"15min":1.129},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":5.7569999999999997,"5min":5.7569999999999997,"15min":5.7569999999999997},"last":1.139},{"interface":"front","average":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":2.9830000000000001,"5min":2.9830000000000001,"15min":2.9830000000000001},"last":0.999}]},{"osd":4,"last update":"Wed Sep 18 10:33:17 2024","interfaces":[{"interface":"back","average":{"1min":1.1000000000000001,"5min":1.1000000000000001,"15min":1.1000000000000001},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":3.8700000000000001,"5min":3.8700000000000001,"15min":3.8700000000000001},"last":1.1899999999999999},{"interface":"front","average":{"1min":1.1359999999999999,"5min":1.1359999999999999,"15min":1.1359999999999999},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":2.6520000000000001,"5min":2.6520000000000001,"15min":2.6520000000000001},"last":1.296}]},{"osd":5,"last update":"Wed Sep 18 10:33:17 2024","interfaces":[{"interface":"back","average":{"1min":1.1160000000000001,"5min":1.1160000000000001,"15min":1.1160000000000001},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":2.0470000000000002,"5min":2.0470000000000002,"15min":2.0470000000000002},"last":1.21},{"interface":"front","average":{"1min":1.0920000000000001,"5min":1.0920000000000001,"15min":1.0920000000000001},"min":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"max":{"1min":1.6679999999999999,"5min":1.6679999999999999,"15min":1.6679999999999999},"last":1.4059999999999999}]},{"osd":6,"last update":"Wed Sep 18 10:33:17 2024","interfaces":[{"interface":"back","average":{"1min":1.3280000000000001,"5min":1.3280000000000001,"15min":1.3280000000000001},"min":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":3.1299999999999999,"5min":3.1299999999999999,"15min":3.1299999999999999},"last":1.5489999999999999},{"interface":"front","average":{"1min":1.123,"5min":1.123,"15min":1.123},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.9390000000000001,"5min":1.9390000000000001,"15min":1.9390000000000001},"last":1.3939999999999999}]},{"osd":7,"last update":"Wed Sep 18 10:33:17 2024","interfaces":[{"interface":"back","average":{"1min":1.1060000000000001,"5min":1.1060000000000001,"15min":1.1060000000000001},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":2.4900000000000002,"5min":2.4900000000000002,"15min":2.4900000000000002},"last":1.4710000000000001},{"interface":"front","average":{"1min":1.3120000000000001,"5min":1.3120000000000001,"15min":1.3120000000000001},"min":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":3.2810000000000001,"5min":3.2810000000000001,"15min":3.2810000000000001},"last":1.5109999999999999}]},{"osd":8,"last update":"Wed Sep 18 10:33:17 2024","interfaces":[{"interface":"back","average":{"1min":1.258,"5min":1.258,"15min":1.258},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":4.4969999999999999,"5min":4.4969999999999999,"15min":4.4969999999999999},"last":1.4850000000000001},{"interface":"front","average":{"1min":1.1419999999999999,"5min":1.1419999999999999,"15min":1.1419999999999999},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":2.1629999999999998,"5min":2.1629999999999998,"15min":2.1629999999999998},"last":1.569}]},{"osd":9,"last update":"Wed Sep 18 10:33:17 2024","interfaces":[{"interface":"back","average":{"1min":1.4419999999999999,"5min":1.4419999999999999,"15min":1.4419999999999999},"min":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"max":{"1min":3.2160000000000002,"5min":3.2160000000000002,"15min":3.2160000000000002},"last":1.601},{"interface":"front","average":{"1min":1.2410000000000001,"5min":1.2410000000000001,"15min":1.2410000000000001},"min":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"max":{"1min":1.9430000000000001,"5min":1.9430000000000001,"15min":1.9430000000000001},"last":1.619}]}]},{"osd":0,"up_from":0,"seq":24,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":298000,"kb_used_data":636,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":104559600,"statfs":{"total":107374182400,"available":107069030400,"internally_reserved":0,"allocated":651264,"data_stored":308977,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[2,3,4,5,6,7,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Sep 18 10:33:15 2024","interfaces":[{"interface":"back","average":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":1.5880000000000001,"5min":1.5880000000000001,"15min":1.5880000000000001},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.917,"5min":1.917,"15min":1.917},"last":1.0600000000000001}]},{"osd":2,"last update":"Wed Sep 18 10:33:15 2024","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.746,"15min":0.746},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.454,"5min":1.454,"15min":1.454},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"min":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.222,"5min":1.222,"15min":1.222},"last":0.81699999999999995}]},{"osd":3,"last update":"Wed Sep 18 10:33:15 2024","interfaces":[{"interface":"back","average":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":1.5509999999999999,"5min":1.5509999999999999,"15min":1.5509999999999999},"last":1.036},{"interface":"front","average":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.6339999999999999,"5min":1.6339999999999999,"15min":1.6339999999999999},"last":0.80200000000000005}]},{"osd":4,"last update":"Wed Sep 18 10:33:15 2024","interfaces":[{"interface":"back","average":{"1min":1,"5min":1,"15min":1},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":1.6339999999999999,"5min":1.6339999999999999,"15min":1.6339999999999999},"last":0.98999999999999999},{"interface":"front","average":{"1min":1.093,"5min":1.093,"15min":1.093},"min":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.704,"5min":1.704,"15min":1.704},"last":1.135}]},{"osd":5,"last update":"Wed Sep 18 10:33:15 2024","interfaces":[{"interface":"back","average":{"1min":1.1699999999999999,"5min":1.1699999999999999,"15min":1.1699999999999999},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.9590000000000001,"5min":1.9590000000000001,"15min":1.9590000000000001},"last":1.1599999999999999},{"interface":"front","average":{"1min":1.2050000000000001,"5min":1.2050000000000001,"15min":1.2050000000000001},"min":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"max":{"1min":2.206,"5min":2.206,"15min":2.206},"last":1.0800000000000001}]},{"osd":6,"last update":"Wed Sep 18 10:33:15 2024","interfaces":[{"interface":"back","average":{"1min":1.157,"5min":1.157,"15min":1.157},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":1.74,"5min":1.74,"15min":1.74},"last":1.4159999999999999},{"interface":"front","average":{"1min":1.135,"5min":1.135,"15min":1.135},"min":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"max":{"1min":2.8140000000000001,"5min":2.8140000000000001,"15min":2.8140000000000001},"last":1.3919999999999999}]},{"osd":7,"last update":"Wed Sep 18 10:33:15 2024","interfaces":[{"interface":"back","average":{"1min":1.268,"5min":1.268,"15min":1.268},"min":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"max":{"1min":2.004,"5min":2.004,"15min":2.004},"last":2.6909999999999998},{"interface":"front","average":{"1min":1.357,"5min":1.357,"15min":1.357},"min":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"max":{"1min":2.7250000000000001,"5min":2.7250000000000001,"15min":2.7250000000000001},"last":1.8899999999999999}]},{"osd":8,"last update":"Wed Sep 18 10:33:15 2024","interfaces":[{"interface":"back","average":{"1min":1.3049999999999999,"5min":1.3049999999999999,"15min":1.3049999999999999},"min":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":2.1419999999999999,"5min":2.1419999999999999,"15min":2.1419999999999999},"last":2.5499999999999998},{"interface":"front","average":{"1min":1.3779999999999999,"5min":1.3779999999999999,"15min":1.3779999999999999},"min":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"max":{"1min":2.8980000000000001,"5min":2.8980000000000001,"15min":2.8980000000000001},"last":1.6279999999999999}]},{"osd":9,"last update":"Wed Sep 18 10:33:15 2024","interfaces":[{"interface":"back","average":{"1min":1.2989999999999999,"5min":1.2989999999999999,"15min":1.2989999999999999},"min":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"max":{"1min":2.3109999999999999,"5min":2.3109999999999999,"15min":2.3109999999999999},"last":1.8140000000000001},{"interface":"front","average":{"1min":1.333,"5min":1.333,"15min":1.333},"min":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":2.2639999999999998,"5min":2.2639999999999998,"15min":2.2639999999999998},"last":2.7719999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-18T10:34:03.939 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-18T10:34:04.230 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-18T10:34:04.233 INFO:teuthology.orchestra.run.smithi134.stderr:dumped all 2024-09-18T10:34:04.245 INFO:teuthology.orchestra.run.smithi134.stdout:{"pg_ready":true,"pg_map":{"version":105,"stamp":"2024-09-18T10:34:03.431290+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":12,"acting":12,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":10,"num_per_pool_osds":10,"num_per_pool_omap_osds":8,"kb":1048576000,"kb_used":2979912,"kb_used_data":6308,"kb_used_omap":16,"kb_used_meta":2973423,"kb_avail":1045596088,"statfs":{"total":1073741824000,"available":1070690394112,"internally_reserved":0,"allocated":6459392,"data_stored":3051408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":17040,"internal_metadata":3044785520},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000990"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":45,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-18T10:33:59.097752+0000","last_change":"2024-09-18T10:33:52.036028+0000","last_active":"2024-09-18T10:33:59.097752+0000","last_peered":"2024-09-18T10:33:59.097752+0000","last_clean":"2024-09-18T10:33:59.097752+0000","last_became_active":"2024-09-18T10:33:52.035591+0000","last_became_peered":"2024-09-18T10:33:52.035591+0000","last_unstale":"2024-09-18T10:33:59.097752+0000","last_undegraded":"2024-09-18T10:33:59.097752+0000","last_fullsized":"2024-09-18T10:33:59.097752+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T10:32:19.443095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:32:19.443095+0000","last_clean_scrub_stamp":"2024-09-18T10:32:19.443095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T18:06:23.838644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00038275899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":61,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-18T10:33:59.101190+0000","last_change":"2024-09-18T10:33:52.042125+0000","last_active":"2024-09-18T10:33:59.101190+0000","last_peered":"2024-09-18T10:33:59.101190+0000","last_clean":"2024-09-18T10:33:59.101190+0000","last_became_active":"2024-09-18T10:33:52.041895+0000","last_became_peered":"2024-09-18T10:33:52.041895+0000","last_unstale":"2024-09-18T10:33:59.101190+0000","last_undegraded":"2024-09-18T10:33:59.101190+0000","last_fullsized":"2024-09-18T10:33:59.101190+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T10:32:19.443095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:32:19.443095+0000","last_clean_scrub_stamp":"2024-09-18T10:32:19.443095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T15:18:02.048451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034975400000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":85,"reported_epoch":57,"state":"active+undersized","last_fresh":"2024-09-18T10:34:00.271592+0000","last_change":"2024-09-18T10:34:00.271514+0000","last_active":"2024-09-18T10:34:00.271592+0000","last_peered":"2024-09-18T10:34:00.271592+0000","last_clean":"2024-09-18T10:33:56.981057+0000","last_became_active":"2024-09-18T10:33:59.102191+0000","last_became_peered":"2024-09-18T10:33:59.102191+0000","last_unstale":"2024-09-18T10:34:00.271592+0000","last_undegraded":"2024-09-18T10:34:00.271592+0000","last_fullsized":"2024-09-18T10:33:59.101046+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T10:32:19.443095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:32:19.443095+0000","last_clean_scrub_stamp":"2024-09-18T10:32:19.443095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T13:05:55.380579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048480000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9],"acting":[9],"avail_no_missing":["9"],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":45,"reported_epoch":56,"state":"active+undersized+wait","last_fresh":"2024-09-18T10:33:59.102960+0000","last_change":"2024-09-18T10:33:59.102253+0000","last_active":"2024-09-18T10:33:59.102960+0000","last_peered":"2024-09-18T10:33:59.102960+0000","last_clean":"2024-09-18T10:33:47.951434+0000","last_became_active":"2024-09-18T10:33:59.102253+0000","last_became_peered":"2024-09-18T10:33:59.102253+0000","last_unstale":"2024-09-18T10:33:59.102960+0000","last_undegraded":"2024-09-18T10:33:59.102960+0000","last_fullsized":"2024-09-18T10:33:59.100957+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T10:32:19.443095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:32:19.443095+0000","last_clean_scrub_stamp":"2024-09-18T10:32:19.443095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T19:47:09.963588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6],"acting":[6],"avail_no_missing":["6"],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.3","version":"18'1","reported_seq":70,"reported_epoch":57,"state":"active+undersized+degraded+wait","last_fresh":"2024-09-18T10:34:00.106984+0000","last_change":"2024-09-18T10:34:00.106902+0000","last_active":"2024-09-18T10:34:00.106984+0000","last_peered":"2024-09-18T10:34:00.106984+0000","last_clean":"2024-09-18T10:33:59.061965+0000","last_became_active":"2024-09-18T10:34:00.106902+0000","last_became_peered":"2024-09-18T10:34:00.106902+0000","last_unstale":"2024-09-18T10:34:00.106984+0000","last_undegraded":"2024-09-18T10:34:00.106209+0000","last_fullsized":"2024-09-18T10:34:00.106121+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T10:32:19.443095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:32:19.443095+0000","last_clean_scrub_stamp":"2024-09-18T10:32:19.443095+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T21:39:17.714747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3],"acting":[3],"avail_no_missing":["3"],"object_location_counts":[{"shards":"3","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.2","version":"20'2","reported_seq":73,"reported_epoch":56,"state":"active+undersized+degraded+wait","last_fresh":"2024-09-18T10:33:59.102609+0000","last_change":"2024-09-18T10:33:59.102002+0000","last_active":"2024-09-18T10:33:59.102609+0000","last_peered":"2024-09-18T10:33:59.102609+0000","last_clean":"2024-09-18T10:33:58.228743+0000","last_became_active":"2024-09-18T10:33:59.102002+0000","last_became_peered":"2024-09-18T10:33:59.102002+0000","last_unstale":"2024-09-18T10:33:59.102609+0000","last_undegraded":"2024-09-18T10:33:59.100769+0000","last_fullsized":"2024-09-18T10:33:59.100508+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T10:32:19.443095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:32:19.443095+0000","last_clean_scrub_stamp":"2024-09-18T10:32:19.443095+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T19:07:49.365548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019475040000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4],"acting":[4],"avail_no_missing":["4"],"object_location_counts":[{"shards":"4","objects":1}],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":42,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-18T10:33:59.061961+0000","last_change":"2024-09-18T10:33:52.038532+0000","last_active":"2024-09-18T10:33:59.061961+0000","last_peered":"2024-09-18T10:33:59.061961+0000","last_clean":"2024-09-18T10:33:59.061961+0000","last_became_active":"2024-09-18T10:33:52.038239+0000","last_became_peered":"2024-09-18T10:33:52.038239+0000","last_unstale":"2024-09-18T10:33:59.061961+0000","last_undegraded":"2024-09-18T10:33:59.061961+0000","last_fullsized":"2024-09-18T10:33:59.061961+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T10:32:19.443095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:32:19.443095+0000","last_clean_scrub_stamp":"2024-09-18T10:32:19.443095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T18:34:12.717302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062722600000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":62,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-18T10:33:59.400931+0000","last_change":"2024-09-18T10:33:52.037375+0000","last_active":"2024-09-18T10:33:59.400931+0000","last_peered":"2024-09-18T10:33:59.400931+0000","last_clean":"2024-09-18T10:33:59.400931+0000","last_became_active":"2024-09-18T10:33:52.037028+0000","last_became_peered":"2024-09-18T10:33:52.037028+0000","last_unstale":"2024-09-18T10:33:59.400931+0000","last_undegraded":"2024-09-18T10:33:59.400931+0000","last_fullsized":"2024-09-18T10:33:59.400931+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T10:32:19.443095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:32:19.443095+0000","last_clean_scrub_stamp":"2024-09-18T10:32:19.443095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T19:41:50.364436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9],"acting":[5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":12,"acting":12,"num_store_stats":9}],"osd_stats":[{"osd":9,"up_from":41,"seq":176093659159,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":298004,"kb_used_data":648,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":104559596,"statfs":{"total":107374182400,"available":107069026304,"internally_reserved":0,"allocated":663552,"data_stored":316651,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[2,3,4,5,6,7,8],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 10:33:18 2024","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":1.1930000000000001,"5min":1.1930000000000001,"15min":1.1930000000000001},"last":1.121},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.3029999999999999,"5min":1.3029999999999999,"15min":1.3029999999999999},"last":0.64900000000000002}]},{"osd":1,"last update":"Wed Sep 18 10:33:18 2024","interfaces":[{"interface":"back","average":{"1min":0.873,"5min":0.873,"15min":0.873},"min":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.3919999999999999,"5min":1.3919999999999999,"15min":1.3919999999999999},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":2.4359999999999999,"5min":2.4359999999999999,"15min":2.4359999999999999},"last":1.6379999999999999}]},{"osd":2,"last update":"Wed Sep 18 10:33:18 2024","interfaces":[{"interface":"back","average":{"1min":1.03,"5min":1.03,"15min":1.03},"min":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":1.526,"5min":1.526,"15min":1.526},"last":0.45300000000000001},{"interface":"front","average":{"1min":1.044,"5min":1.044,"15min":1.044},"min":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"max":{"1min":1.591,"5min":1.591,"15min":1.591},"last":0.52000000000000002}]},{"osd":3,"last update":"Wed Sep 18 10:33:18 2024","interfaces":[{"interface":"back","average":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.542,"5min":1.542,"15min":1.542},"last":0.63800000000000001},{"interface":"front","average":{"1min":1.274,"5min":1.274,"15min":1.274},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.8440000000000001,"5min":1.8440000000000001,"15min":1.8440000000000001},"last":0.98099999999999998}]},{"osd":4,"last update":"Wed Sep 18 10:33:18 2024","interfaces":[{"interface":"back","average":{"1min":1.2430000000000001,"5min":1.2430000000000001,"15min":1.2430000000000001},"min":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"max":{"1min":1.8180000000000001,"5min":1.8180000000000001,"15min":1.8180000000000001},"last":0.67300000000000004},{"interface":"front","average":{"1min":1.276,"5min":1.276,"15min":1.276},"min":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"max":{"1min":1.681,"5min":1.681,"15min":1.681},"last":0.96399999999999997}]},{"osd":5,"last update":"Wed Sep 18 10:33:18 2024","interfaces":[{"interface":"back","average":{"1min":1.3140000000000001,"5min":1.3140000000000001,"15min":1.3140000000000001},"min":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"max":{"1min":1.8600000000000001,"5min":1.8600000000000001,"15min":1.8600000000000001},"last":1.0269999999999999},{"interface":"front","average":{"1min":1.3009999999999999,"5min":1.3009999999999999,"15min":1.3009999999999999},"min":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"max":{"1min":1.7090000000000001,"5min":1.7090000000000001,"15min":1.7090000000000001},"last":0.78100000000000003}]},{"osd":6,"last update":"Wed Sep 18 10:33:18 2024","interfaces":[{"interface":"back","average":{"1min":1.3029999999999999,"5min":1.3029999999999999,"15min":1.3029999999999999},"min":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":2.778,"5min":2.778,"15min":2.778},"last":0.86599999999999999},{"interface":"front","average":{"1min":1.345,"5min":1.345,"15min":1.345},"min":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":1.921,"5min":1.921,"15min":1.921},"last":1.075}]},{"osd":7,"last update":"Wed Sep 18 10:33:18 2024","interfaces":[{"interface":"back","average":{"1min":1.3340000000000001,"5min":1.3340000000000001,"15min":1.3340000000000001},"min":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"max":{"1min":1.6830000000000001,"5min":1.6830000000000001,"15min":1.6830000000000001},"last":1.1439999999999999},{"interface":"front","average":{"1min":1.417,"5min":1.417,"15min":1.417},"min":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"max":{"1min":3.0390000000000001,"5min":3.0390000000000001,"15min":3.0390000000000001},"last":0.90600000000000003}]},{"osd":8,"last update":"Wed Sep 18 10:33:18 2024","interfaces":[{"interface":"back","average":{"1min":1.4330000000000001,"5min":1.4330000000000001,"15min":1.4330000000000001},"min":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"max":{"1min":2.8500000000000001,"5min":2.8500000000000001,"15min":2.8500000000000001},"last":0.93999999999999995},{"interface":"front","average":{"1min":1.4079999999999999,"5min":1.4079999999999999,"15min":1.4079999999999999},"min":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"max":{"1min":2.0070000000000001,"5min":2.0070000000000001,"15min":2.0070000000000001},"last":1.1910000000000001}]}]},{"osd":8,"up_from":48,"seq":206158430232,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":104857600,"kb_used":297956,"kb_used_data":600,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":104559644,"statfs":{"total":107374182400,"available":107069075456,"internally_reserved":0,"allocated":614400,"data_stored":285923,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,6,7,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 10:33:16 2024","interfaces":[{"interface":"back","average":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":1.26,"5min":1.26,"15min":1.26},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":1.238,"5min":1.238,"15min":1.238},"last":0.86299999999999999}]},{"osd":1,"last update":"Wed Sep 18 10:33:16 2024","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.751,"15min":0.751},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":1.129,"5min":1.129,"15min":1.129},"last":1.1359999999999999},{"interface":"front","average":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"min":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.2390000000000001,"5min":1.2390000000000001,"15min":1.2390000000000001},"last":0.92600000000000005}]},{"osd":2,"last update":"Wed Sep 18 10:33:16 2024","interfaces":[{"interface":"back","average":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":1.6439999999999999,"5min":1.6439999999999999,"15min":1.6439999999999999},"last":1.238},{"interface":"front","average":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":1.2529999999999999,"5min":1.2529999999999999,"15min":1.2529999999999999},"last":0.82299999999999995}]},{"osd":3,"last update":"Wed Sep 18 10:33:16 2024","interfaces":[{"interface":"back","average":{"1min":1.109,"5min":1.109,"15min":1.109},"min":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"max":{"1min":1.538,"5min":1.538,"15min":1.538},"last":1.458},{"interface":"front","average":{"1min":1.0389999999999999,"5min":1.0389999999999999,"15min":1.0389999999999999},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.4810000000000001,"5min":1.4810000000000001,"15min":1.4810000000000001},"last":1.581}]},{"osd":4,"last update":"Wed Sep 18 10:33:16 2024","interfaces":[{"interface":"back","average":{"1min":1.0620000000000001,"5min":1.0620000000000001,"15min":1.0620000000000001},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.8169999999999999,"5min":1.8169999999999999,"15min":1.8169999999999999},"last":1.917},{"interface":"front","average":{"1min":1.2549999999999999,"5min":1.2549999999999999,"15min":1.2549999999999999},"min":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"max":{"1min":2.1480000000000001,"5min":2.1480000000000001,"15min":2.1480000000000001},"last":1.3600000000000001}]},{"osd":5,"last update":"Wed Sep 18 10:33:16 2024","interfaces":[{"interface":"back","average":{"1min":1.216,"5min":1.216,"15min":1.216},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":2.3109999999999999,"5min":2.3109999999999999,"15min":2.3109999999999999},"last":1.637},{"interface":"front","average":{"1min":1.161,"5min":1.161,"15min":1.161},"min":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":2.0169999999999999,"5min":2.0169999999999999,"15min":2.0169999999999999},"last":1.7110000000000001}]},{"osd":6,"last update":"Wed Sep 18 10:33:16 2024","interfaces":[{"interface":"back","average":{"1min":1.2889999999999999,"5min":1.2889999999999999,"15min":1.2889999999999999},"min":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"max":{"1min":2.1899999999999999,"5min":2.1899999999999999,"15min":2.1899999999999999},"last":1.9470000000000001},{"interface":"front","average":{"1min":1.1799999999999999,"5min":1.1799999999999999,"15min":1.1799999999999999},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":2.069,"5min":2.069,"15min":2.069},"last":1.764}]},{"osd":7,"last update":"Wed Sep 18 10:33:16 2024","interfaces":[{"interface":"back","average":{"1min":1.2589999999999999,"5min":1.2589999999999999,"15min":1.2589999999999999},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":2.2130000000000001,"5min":2.2130000000000001,"15min":2.2130000000000001},"last":1.5129999999999999},{"interface":"front","average":{"1min":1.4039999999999999,"5min":1.4039999999999999,"15min":1.4039999999999999},"min":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"max":{"1min":2.2450000000000001,"5min":2.2450000000000001,"15min":2.2450000000000001},"last":2.0459999999999998}]},{"osd":9,"last update":"Wed Sep 18 10:33:16 2024","interfaces":[{"interface":"back","average":{"1min":1.304,"5min":1.304,"15min":1.304},"min":{"1min":0.63,"5min":0.63,"15min":0.63},"max":{"1min":2.1800000000000002,"5min":2.1800000000000002,"15min":2.1800000000000002},"last":2.0790000000000002},{"interface":"front","average":{"1min":1.3440000000000001,"5min":1.3440000000000001,"15min":1.3440000000000001},"min":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"max":{"1min":2.3969999999999998,"5min":2.3969999999999998,"15min":2.3969999999999998},"last":1.875}]}]},{"osd":7,"up_from":48,"seq":206158430231,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":297968,"kb_used_data":612,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":104559632,"statfs":{"total":107374182400,"available":107069063168,"internally_reserved":0,"allocated":626688,"data_stored":293647,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,6,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 10:33:17 2024","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.202,"5min":1.202,"15min":1.202},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.75,"5min":0.75,"15min":0.75},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":1.486,"5min":1.486,"15min":1.486},"last":0.98799999999999999}]},{"osd":1,"last update":"Wed Sep 18 10:33:17 2024","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.3009999999999999,"5min":1.3009999999999999,"15min":1.3009999999999999},"last":1.085},{"interface":"front","average":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.8480000000000001,"5min":1.8480000000000001,"15min":1.8480000000000001},"last":0.73799999999999999}]},{"osd":2,"last update":"Wed Sep 18 10:33:17 2024","interfaces":[{"interface":"back","average":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":1.5980000000000001,"5min":1.5980000000000001,"15min":1.5980000000000001},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.3440000000000001,"5min":1.3440000000000001,"15min":1.3440000000000001},"last":0.77800000000000002}]},{"osd":3,"last update":"Wed Sep 18 10:33:17 2024","interfaces":[{"interface":"back","average":{"1min":1.1000000000000001,"5min":1.1000000000000001,"15min":1.1000000000000001},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.9319999999999999,"5min":1.9319999999999999,"15min":1.9319999999999999},"last":0.85099999999999998},{"interface":"front","average":{"1min":1.0720000000000001,"5min":1.0720000000000001,"15min":1.0720000000000001},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.8879999999999999,"5min":1.8879999999999999,"15min":1.8879999999999999},"last":0.91400000000000003}]},{"osd":4,"last update":"Wed Sep 18 10:33:17 2024","interfaces":[{"interface":"back","average":{"1min":1.097,"5min":1.097,"15min":1.097},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":2.1379999999999999,"5min":2.1379999999999999,"15min":2.1379999999999999},"last":1.8680000000000001},{"interface":"front","average":{"1min":1.1060000000000001,"5min":1.1060000000000001,"15min":1.1060000000000001},"min":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":2.0110000000000001,"5min":2.0110000000000001,"15min":2.0110000000000001},"last":0.90400000000000003}]},{"osd":5,"last update":"Wed Sep 18 10:33:17 2024","interfaces":[{"interface":"back","average":{"1min":1.194,"5min":1.194,"15min":1.194},"min":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"max":{"1min":2.4889999999999999,"5min":2.4889999999999999,"15min":2.4889999999999999},"last":1.0329999999999999},{"interface":"front","average":{"1min":1.2230000000000001,"5min":1.2230000000000001,"15min":1.2230000000000001},"min":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"max":{"1min":2.4289999999999998,"5min":2.4289999999999998,"15min":2.4289999999999998},"last":2.056}]},{"osd":6,"last update":"Wed Sep 18 10:33:17 2024","interfaces":[{"interface":"back","average":{"1min":1.2769999999999999,"5min":1.2769999999999999,"15min":1.2769999999999999},"min":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"max":{"1min":2.0950000000000002,"5min":2.0950000000000002,"15min":2.0950000000000002},"last":1.341},{"interface":"front","average":{"1min":1.218,"5min":1.218,"15min":1.218},"min":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":1.9450000000000001,"5min":1.9450000000000001,"15min":1.9450000000000001},"last":1.1930000000000001}]},{"osd":8,"last update":"Wed Sep 18 10:33:17 2024","interfaces":[{"interface":"back","average":{"1min":1.2969999999999999,"5min":1.2969999999999999,"15min":1.2969999999999999},"min":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"max":{"1min":2.6389999999999998,"5min":2.6389999999999998,"15min":2.6389999999999998},"last":1.9810000000000001},{"interface":"front","average":{"1min":1.3600000000000001,"5min":1.3600000000000001,"15min":1.3600000000000001},"min":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"max":{"1min":2.1520000000000001,"5min":2.1520000000000001,"15min":2.1520000000000001},"last":1.2789999999999999}]},{"osd":9,"last update":"Wed Sep 18 10:33:17 2024","interfaces":[{"interface":"back","average":{"1min":1.2470000000000001,"5min":1.2470000000000001,"15min":1.2470000000000001},"min":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":2.0680000000000001,"5min":2.0680000000000001,"15min":2.0680000000000001},"last":2.2469999999999999},{"interface":"front","average":{"1min":1.282,"5min":1.282,"15min":1.282},"min":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"max":{"1min":1.8460000000000001,"5min":1.8460000000000001,"15min":1.8460000000000001},"last":2.9929999999999999}]}]},{"osd":6,"up_from":48,"seq":206158430231,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":297988,"kb_used_data":624,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":104559612,"statfs":{"total":107374182400,"available":107069042688,"internally_reserved":0,"allocated":638976,"data_stored":301295,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,7,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 10:33:17 2024","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":1.073,"5min":1.073,"15min":1.073},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":1.3720000000000001,"5min":1.3720000000000001,"15min":1.3720000000000001},"last":0.70499999999999996}]},{"osd":1,"last update":"Wed Sep 18 10:33:17 2024","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.751,"15min":0.751},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":1.6599999999999999,"5min":1.6599999999999999,"15min":1.6599999999999999},"last":1.095},{"interface":"front","average":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":2.0459999999999998,"5min":2.0459999999999998,"15min":2.0459999999999998},"last":0.61399999999999999}]},{"osd":2,"last update":"Wed Sep 18 10:33:17 2024","interfaces":[{"interface":"back","average":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":1.1399999999999999,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":0.77000000000000002},{"interface":"front","average":{"1min":1.052,"5min":1.052,"15min":1.052},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":1.907,"5min":1.907,"15min":1.907},"last":0.92400000000000004}]},{"osd":3,"last update":"Wed Sep 18 10:33:17 2024","interfaces":[{"interface":"back","average":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.7490000000000001,"5min":1.7490000000000001,"15min":1.7490000000000001},"last":0.749},{"interface":"front","average":{"1min":1.1830000000000001,"5min":1.1830000000000001,"15min":1.1830000000000001},"min":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"max":{"1min":1.8680000000000001,"5min":1.8680000000000001,"15min":1.8680000000000001},"last":1.1679999999999999}]},{"osd":4,"last update":"Wed Sep 18 10:33:17 2024","interfaces":[{"interface":"back","average":{"1min":1.2070000000000001,"5min":1.2070000000000001,"15min":1.2070000000000001},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.976,"5min":1.976,"15min":1.976},"last":1.25},{"interface":"front","average":{"1min":1.127,"5min":1.127,"15min":1.127},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":2.544,"5min":2.544,"15min":2.544},"last":1.6200000000000001}]},{"osd":5,"last update":"Wed Sep 18 10:33:17 2024","interfaces":[{"interface":"back","average":{"1min":1.2170000000000001,"5min":1.2170000000000001,"15min":1.2170000000000001},"min":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"max":{"1min":2.1539999999999999,"5min":2.1539999999999999,"15min":2.1539999999999999},"last":1.698},{"interface":"front","average":{"1min":1.296,"5min":1.296,"15min":1.296},"min":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":2.2469999999999999,"5min":2.2469999999999999,"15min":2.2469999999999999},"last":1.5649999999999999}]},{"osd":7,"last update":"Wed Sep 18 10:33:17 2024","interfaces":[{"interface":"back","average":{"1min":1.246,"5min":1.246,"15min":1.246},"min":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":2.6440000000000001,"5min":2.6440000000000001,"15min":2.6440000000000001},"last":1.2769999999999999},{"interface":"front","average":{"1min":1.7609999999999999,"5min":1.7609999999999999,"15min":1.7609999999999999},"min":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"max":{"1min":10.385,"5min":10.385,"15min":10.385},"last":1.633}]},{"osd":8,"last update":"Wed Sep 18 10:33:17 2024","interfaces":[{"interface":"back","average":{"1min":1.3640000000000001,"5min":1.3640000000000001,"15min":1.3640000000000001},"min":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"max":{"1min":2.444,"5min":2.444,"15min":2.444},"last":1.516},{"interface":"front","average":{"1min":1.274,"5min":1.274,"15min":1.274},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":2.714,"5min":2.714,"15min":2.714},"last":1.3580000000000001}]},{"osd":9,"last update":"Wed Sep 18 10:33:17 2024","interfaces":[{"interface":"back","average":{"1min":1.2789999999999999,"5min":1.2789999999999999,"15min":1.2789999999999999},"min":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"max":{"1min":2.2879999999999998,"5min":2.2879999999999998,"15min":2.2879999999999998},"last":1.768},{"interface":"front","average":{"1min":1.3240000000000001,"5min":1.3240000000000001,"15min":1.3240000000000001},"min":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":2.5960000000000001,"5min":2.5960000000000001,"15min":2.5960000000000001},"last":1.669}]}]},{"osd":5,"up_from":48,"seq":206158430233,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":298000,"kb_used_data":636,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":104559600,"statfs":{"total":107374182400,"available":107069030400,"internally_reserved":0,"allocated":651264,"data_stored":308977,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[2,3,4,6,7,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 10:33:18 2024","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.111,"5min":1.111,"15min":1.111},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"min":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.5249999999999999,"5min":1.5249999999999999,"15min":1.5249999999999999},"last":0.63}]},{"osd":1,"last update":"Wed Sep 18 10:33:18 2024","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.173,"5min":1.173,"15min":1.173},"last":0.79600000000000004},{"interface":"front","average":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.8280000000000001,"5min":1.8280000000000001,"15min":1.8280000000000001},"last":0.88500000000000001}]},{"osd":2,"last update":"Wed Sep 18 10:33:18 2024","interfaces":[{"interface":"back","average":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"min":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":1.4159999999999999,"5min":1.4159999999999999,"15min":1.4159999999999999},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"min":{"1min":0.627,"5min":0.627,"15min":0.627},"max":{"1min":2.3849999999999998,"5min":2.3849999999999998,"15min":2.3849999999999998},"last":0.63200000000000001}]},{"osd":3,"last update":"Wed Sep 18 10:33:18 2024","interfaces":[{"interface":"back","average":{"1min":1.079,"5min":1.079,"15min":1.079},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.923,"5min":1.923,"15min":1.923},"last":0.57099999999999995},{"interface":"front","average":{"1min":1.268,"5min":1.268,"15min":1.268},"min":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"max":{"1min":2.073,"5min":2.073,"15min":2.073},"last":0.72799999999999998}]},{"osd":4,"last update":"Wed Sep 18 10:33:18 2024","interfaces":[{"interface":"back","average":{"1min":1.123,"5min":1.123,"15min":1.123},"min":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"max":{"1min":2.141,"5min":2.141,"15min":2.141},"last":0.60699999999999998},{"interface":"front","average":{"1min":1.2490000000000001,"5min":1.2490000000000001,"15min":1.2490000000000001},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":2.3300000000000001,"5min":2.3300000000000001,"15min":2.3300000000000001},"last":1.05}]},{"osd":6,"last update":"Wed Sep 18 10:33:18 2024","interfaces":[{"interface":"back","average":{"1min":1.2769999999999999,"5min":1.2769999999999999,"15min":1.2769999999999999},"min":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"max":{"1min":1.806,"5min":1.806,"15min":1.806},"last":0.81499999999999995},{"interface":"front","average":{"1min":1.274,"5min":1.274,"15min":1.274},"min":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"max":{"1min":2.6120000000000001,"5min":2.6120000000000001,"15min":2.6120000000000001},"last":0.70199999999999996}]},{"osd":7,"last update":"Wed Sep 18 10:33:18 2024","interfaces":[{"interface":"back","average":{"1min":1.3380000000000001,"5min":1.3380000000000001,"15min":1.3380000000000001},"min":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"max":{"1min":2.661,"5min":2.661,"15min":2.661},"last":1.018},{"interface":"front","average":{"1min":1.3819999999999999,"5min":1.3819999999999999,"15min":1.3819999999999999},"min":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"max":{"1min":2.548,"5min":2.548,"15min":2.548},"last":0.89600000000000002}]},{"osd":8,"last update":"Wed Sep 18 10:33:18 2024","interfaces":[{"interface":"back","average":{"1min":1.359,"5min":1.359,"15min":1.359},"min":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"max":{"1min":2.7080000000000002,"5min":2.7080000000000002,"15min":2.7080000000000002},"last":0.96599999999999997},{"interface":"front","average":{"1min":1.327,"5min":1.327,"15min":1.327},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":2.4399999999999999,"5min":2.4399999999999999,"15min":2.4399999999999999},"last":1.169}]},{"osd":9,"last update":"Wed Sep 18 10:33:18 2024","interfaces":[{"interface":"back","average":{"1min":1.3859999999999999,"5min":1.3859999999999999,"15min":1.3859999999999999},"min":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"max":{"1min":2.2130000000000001,"5min":2.2130000000000001,"15min":2.2130000000000001},"last":0.92700000000000005},{"interface":"front","average":{"1min":1.4019999999999999,"5min":1.4019999999999999,"15min":1.4019999999999999},"min":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"max":{"1min":2.782,"5min":2.782,"15min":2.782},"last":1.0860000000000001}]}]},{"osd":4,"up_from":48,"seq":206158430231,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":297992,"kb_used_data":628,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":104559608,"statfs":{"total":107374182400,"available":107069038592,"internally_reserved":0,"allocated":643072,"data_stored":301314,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,5,6,7,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 10:33:16 2024","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":1.0960000000000001,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.745,"5min":0.745,"15min":0.745},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.171,"5min":1.171,"15min":1.171},"last":0.70799999999999996}]},{"osd":1,"last update":"Wed Sep 18 10:33:16 2024","interfaces":[{"interface":"back","average":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":1.1579999999999999,"5min":1.1579999999999999,"15min":1.1579999999999999},"last":1.0820000000000001},{"interface":"front","average":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.583,"5min":1.583,"15min":1.583},"last":0.77400000000000002}]},{"osd":2,"last update":"Wed Sep 18 10:33:16 2024","interfaces":[{"interface":"back","average":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":2.1139999999999999,"5min":2.1139999999999999,"15min":2.1139999999999999},"last":1.165},{"interface":"front","average":{"1min":0.995,"5min":0.995,"15min":0.995},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.5660000000000001,"5min":1.5660000000000001,"15min":1.5660000000000001},"last":0.56100000000000005}]},{"osd":3,"last update":"Wed Sep 18 10:33:16 2024","interfaces":[{"interface":"back","average":{"1min":1.0620000000000001,"5min":1.0620000000000001,"15min":1.0620000000000001},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.6200000000000001,"5min":1.6200000000000001,"15min":1.6200000000000001},"last":0.91400000000000003},{"interface":"front","average":{"1min":1.1699999999999999,"5min":1.1699999999999999,"15min":1.1699999999999999},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.732,"5min":1.732,"15min":1.732},"last":0.876}]},{"osd":5,"last update":"Wed Sep 18 10:33:16 2024","interfaces":[{"interface":"back","average":{"1min":1.159,"5min":1.159,"15min":1.159},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":2.2509999999999999,"5min":2.2509999999999999,"15min":2.2509999999999999},"last":1.0489999999999999},{"interface":"front","average":{"1min":1.1830000000000001,"5min":1.1830000000000001,"15min":1.1830000000000001},"min":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":1.873,"5min":1.873,"15min":1.873},"last":1.0049999999999999}]},{"osd":6,"last update":"Wed Sep 18 10:33:16 2024","interfaces":[{"interface":"back","average":{"1min":1.214,"5min":1.214,"15min":1.214},"min":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"max":{"1min":2.2879999999999998,"5min":2.2879999999999998,"15min":2.2879999999999998},"last":1.232},{"interface":"front","average":{"1min":1.25,"5min":1.25,"15min":1.25},"min":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":2.661,"5min":2.661,"15min":2.661},"last":1.137}]},{"osd":7,"last update":"Wed Sep 18 10:33:16 2024","interfaces":[{"interface":"back","average":{"1min":1.319,"5min":1.319,"15min":1.319},"min":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"max":{"1min":1.895,"5min":1.895,"15min":1.895},"last":1.415},{"interface":"front","average":{"1min":1.2450000000000001,"5min":1.2450000000000001,"15min":1.2450000000000001},"min":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":2.0579999999999998,"5min":2.0579999999999998,"15min":2.0579999999999998},"last":1.6619999999999999}]},{"osd":8,"last update":"Wed Sep 18 10:33:16 2024","interfaces":[{"interface":"back","average":{"1min":1.278,"5min":1.278,"15min":1.278},"min":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":2.173,"5min":2.173,"15min":2.173},"last":1.573},{"interface":"front","average":{"1min":1.3240000000000001,"5min":1.3240000000000001,"15min":1.3240000000000001},"min":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":1.9750000000000001,"5min":1.9750000000000001,"15min":1.9750000000000001},"last":1.375}]},{"osd":9,"last update":"Wed Sep 18 10:33:16 2024","interfaces":[{"interface":"back","average":{"1min":1.327,"5min":1.327,"15min":1.327},"min":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":2.448,"5min":2.448,"15min":2.448},"last":1.4419999999999999},{"interface":"front","average":{"1min":1.389,"5min":1.389,"15min":1.389},"min":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"max":{"1min":2.242,"5min":2.242,"15min":2.242},"last":1.474}]}]},{"osd":3,"up_from":48,"seq":206158430233,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":298000,"kb_used_data":636,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":104559600,"statfs":{"total":107374182400,"available":107069030400,"internally_reserved":0,"allocated":651264,"data_stored":308977,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[2,4,5,6,7,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 10:33:15 2024","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.095,"5min":1.095,"15min":1.095},"last":1.4199999999999999},{"interface":"front","average":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"min":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":2.121,"5min":2.121,"15min":2.121},"last":0.84499999999999997}]},{"osd":1,"last update":"Wed Sep 18 10:33:15 2024","interfaces":[{"interface":"back","average":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":1.9259999999999999,"5min":1.9259999999999999,"15min":1.9259999999999999},"last":1.1140000000000001},{"interface":"front","average":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"min":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":1.2090000000000001,"5min":1.2090000000000001,"15min":1.2090000000000001},"last":0.80800000000000005}]},{"osd":2,"last update":"Wed Sep 18 10:33:15 2024","interfaces":[{"interface":"back","average":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.204,"5min":1.204,"15min":1.204},"last":0.90900000000000003},{"interface":"front","average":{"1min":1.1870000000000001,"5min":1.1870000000000001,"15min":1.1870000000000001},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":3.2040000000000002,"5min":3.2040000000000002,"15min":3.2040000000000002},"last":2.6230000000000002}]},{"osd":4,"last update":"Wed Sep 18 10:33:15 2024","interfaces":[{"interface":"back","average":{"1min":1.071,"5min":1.071,"15min":1.071},"min":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":1.49,"5min":1.49,"15min":1.49},"last":0.82799999999999996},{"interface":"front","average":{"1min":1.266,"5min":1.266,"15min":1.266},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":2.7519999999999998,"5min":2.7519999999999998,"15min":2.7519999999999998},"last":1.016}]},{"osd":5,"last update":"Wed Sep 18 10:33:15 2024","interfaces":[{"interface":"back","average":{"1min":1.1830000000000001,"5min":1.1830000000000001,"15min":1.1830000000000001},"min":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"max":{"1min":2.0630000000000002,"5min":2.0630000000000002,"15min":2.0630000000000002},"last":0.71299999999999997},{"interface":"front","average":{"1min":1.1259999999999999,"5min":1.1259999999999999,"15min":1.1259999999999999},"min":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"max":{"1min":1.5149999999999999,"5min":1.5149999999999999,"15min":1.5149999999999999},"last":1.115}]},{"osd":6,"last update":"Wed Sep 18 10:33:15 2024","interfaces":[{"interface":"back","average":{"1min":1.3620000000000001,"5min":1.3620000000000001,"15min":1.3620000000000001},"min":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":4.1029999999999998,"5min":4.1029999999999998,"15min":4.1029999999999998},"last":0.97699999999999998},{"interface":"front","average":{"1min":1.268,"5min":1.268,"15min":1.268},"min":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"max":{"1min":2.2789999999999999,"5min":2.2789999999999999,"15min":2.2789999999999999},"last":1.427}]},{"osd":7,"last update":"Wed Sep 18 10:33:15 2024","interfaces":[{"interface":"back","average":{"1min":1.2010000000000001,"5min":1.2010000000000001,"15min":1.2010000000000001},"min":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"max":{"1min":2.1880000000000002,"5min":2.1880000000000002,"15min":2.1880000000000002},"last":2.7589999999999999},{"interface":"front","average":{"1min":1.282,"5min":1.282,"15min":1.282},"min":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"max":{"1min":3.0409999999999999,"5min":3.0409999999999999,"15min":3.0409999999999999},"last":1.2}]},{"osd":8,"last update":"Wed Sep 18 10:33:15 2024","interfaces":[{"interface":"back","average":{"1min":1.5109999999999999,"5min":1.5109999999999999,"15min":1.5109999999999999},"min":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"max":{"1min":3.5590000000000002,"5min":3.5590000000000002,"15min":3.5590000000000002},"last":1.3360000000000001},{"interface":"front","average":{"1min":1.3540000000000001,"5min":1.3540000000000001,"15min":1.3540000000000001},"min":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"max":{"1min":2.3919999999999999,"5min":2.3919999999999999,"15min":2.3919999999999999},"last":1.4850000000000001}]},{"osd":9,"last update":"Wed Sep 18 10:33:15 2024","interfaces":[{"interface":"back","average":{"1min":1.379,"5min":1.379,"15min":1.379},"min":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"max":{"1min":3.1259999999999999,"5min":3.1259999999999999,"15min":3.1259999999999999},"last":1.522},{"interface":"front","average":{"1min":1.5109999999999999,"5min":1.5109999999999999,"15min":1.5109999999999999},"min":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"max":{"1min":3.6339999999999999,"5min":3.6339999999999999,"15min":3.6339999999999999},"last":1.6020000000000001}]}]},{"osd":2,"up_from":48,"seq":206158430231,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":104857600,"kb_used":297996,"kb_used_data":636,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":104559604,"statfs":{"total":107374182400,"available":107069034496,"internally_reserved":0,"allocated":651264,"data_stored":308977,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,3,4,5,6,7,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 10:33:18 2024","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1.109,"5min":1.109,"15min":1.109},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":1.5029999999999999,"5min":1.5029999999999999,"15min":1.5029999999999999},"last":0.69299999999999995}]},{"osd":1,"last update":"Wed Sep 18 10:33:18 2024","interfaces":[{"interface":"back","average":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"min":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.651,"5min":1.651,"15min":1.651},"last":0.71799999999999997},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":1.1559999999999999,"5min":1.1559999999999999,"15min":1.1559999999999999},"last":1.1599999999999999}]},{"osd":3,"last update":"Wed Sep 18 10:33:18 2024","interfaces":[{"interface":"back","average":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":1.2869999999999999,"5min":1.2869999999999999,"15min":1.2869999999999999},"last":0.92800000000000005},{"interface":"front","average":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.7390000000000001,"5min":1.7390000000000001,"15min":1.7390000000000001},"last":0.83899999999999997}]},{"osd":4,"last update":"Wed Sep 18 10:33:18 2024","interfaces":[{"interface":"back","average":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"min":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":1.333,"5min":1.333,"15min":1.333},"last":1.329},{"interface":"front","average":{"1min":1.151,"5min":1.151,"15min":1.151},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.7869999999999999,"5min":1.7869999999999999,"15min":1.7869999999999999},"last":0.89300000000000002}]},{"osd":5,"last update":"Wed Sep 18 10:33:18 2024","interfaces":[{"interface":"back","average":{"1min":1.1100000000000001,"5min":1.1100000000000001,"15min":1.1100000000000001},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.597,"5min":1.597,"15min":1.597},"last":1.2909999999999999},{"interface":"front","average":{"1min":1.246,"5min":1.246,"15min":1.246},"min":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"max":{"1min":1.873,"5min":1.873,"15min":1.873},"last":1.0029999999999999}]},{"osd":6,"last update":"Wed Sep 18 10:33:18 2024","interfaces":[{"interface":"back","average":{"1min":1.0649999999999999,"5min":1.0649999999999999,"15min":1.0649999999999999},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":1.696,"5min":1.696,"15min":1.696},"last":1.6259999999999999},{"interface":"front","average":{"1min":1.1479999999999999,"5min":1.1479999999999999,"15min":1.1479999999999999},"min":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"max":{"1min":1.5289999999999999,"5min":1.5289999999999999,"15min":1.5289999999999999},"last":1.605}]},{"osd":7,"last update":"Wed Sep 18 10:33:18 2024","interfaces":[{"interface":"back","average":{"1min":1.268,"5min":1.268,"15min":1.268},"min":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":1.9350000000000001,"5min":1.9350000000000001,"15min":1.9350000000000001},"last":1.492},{"interface":"front","average":{"1min":1.244,"5min":1.244,"15min":1.244},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":2.6419999999999999,"5min":2.6419999999999999,"15min":2.6419999999999999},"last":1.534}]},{"osd":8,"last update":"Wed Sep 18 10:33:18 2024","interfaces":[{"interface":"back","average":{"1min":1.306,"5min":1.306,"15min":1.306},"min":{"1min":1,"5min":1,"15min":1},"max":{"1min":1.613,"5min":1.613,"15min":1.613},"last":1.093},{"interface":"front","average":{"1min":1.278,"5min":1.278,"15min":1.278},"min":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":1.9830000000000001,"5min":1.9830000000000001,"15min":1.9830000000000001},"last":1.7}]},{"osd":9,"last update":"Wed Sep 18 10:33:18 2024","interfaces":[{"interface":"back","average":{"1min":1.2210000000000001,"5min":1.2210000000000001,"15min":1.2210000000000001},"min":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"max":{"1min":2.4220000000000002,"5min":2.4220000000000002,"15min":2.4220000000000002},"last":1.5720000000000001},{"interface":"front","average":{"1min":1.337,"5min":1.337,"15min":1.337},"min":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"max":{"1min":2.1349999999999998,"5min":2.1349999999999998,"15min":2.1349999999999998},"last":1.0609999999999999}]}]},{"osd":1,"up_from":0,"seq":24,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":298008,"kb_used_data":652,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":104559592,"statfs":{"total":107374182400,"available":107069022208,"internally_reserved":0,"allocated":667648,"data_stored":316670,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[2,3,4,5,6,7,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 10:33:17 2024","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":1.0980000000000001,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.89900000000000002},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"min":{"1min":0.248,"5min":0.248,"15min":0.248},"max":{"1min":1.3,"5min":1.3,"15min":1.3},"last":0.58699999999999997}]},{"osd":2,"last update":"Wed Sep 18 10:33:17 2024","interfaces":[{"interface":"back","average":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.206,"5min":1.206,"15min":1.206},"last":1.1020000000000001},{"interface":"front","average":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":1.54,"5min":1.54,"15min":1.54},"last":0.83199999999999996}]},{"osd":3,"last update":"Wed Sep 18 10:33:17 2024","interfaces":[{"interface":"back","average":{"1min":1.129,"5min":1.129,"15min":1.129},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":5.7569999999999997,"5min":5.7569999999999997,"15min":5.7569999999999997},"last":1.139},{"interface":"front","average":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":2.9830000000000001,"5min":2.9830000000000001,"15min":2.9830000000000001},"last":0.999}]},{"osd":4,"last update":"Wed Sep 18 10:33:17 2024","interfaces":[{"interface":"back","average":{"1min":1.1000000000000001,"5min":1.1000000000000001,"15min":1.1000000000000001},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":3.8700000000000001,"5min":3.8700000000000001,"15min":3.8700000000000001},"last":1.1899999999999999},{"interface":"front","average":{"1min":1.1359999999999999,"5min":1.1359999999999999,"15min":1.1359999999999999},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":2.6520000000000001,"5min":2.6520000000000001,"15min":2.6520000000000001},"last":1.296}]},{"osd":5,"last update":"Wed Sep 18 10:33:17 2024","interfaces":[{"interface":"back","average":{"1min":1.1160000000000001,"5min":1.1160000000000001,"15min":1.1160000000000001},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":2.0470000000000002,"5min":2.0470000000000002,"15min":2.0470000000000002},"last":1.21},{"interface":"front","average":{"1min":1.0920000000000001,"5min":1.0920000000000001,"15min":1.0920000000000001},"min":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"max":{"1min":1.6679999999999999,"5min":1.6679999999999999,"15min":1.6679999999999999},"last":1.4059999999999999}]},{"osd":6,"last update":"Wed Sep 18 10:33:17 2024","interfaces":[{"interface":"back","average":{"1min":1.3280000000000001,"5min":1.3280000000000001,"15min":1.3280000000000001},"min":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":3.1299999999999999,"5min":3.1299999999999999,"15min":3.1299999999999999},"last":1.5489999999999999},{"interface":"front","average":{"1min":1.123,"5min":1.123,"15min":1.123},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.9390000000000001,"5min":1.9390000000000001,"15min":1.9390000000000001},"last":1.3939999999999999}]},{"osd":7,"last update":"Wed Sep 18 10:33:17 2024","interfaces":[{"interface":"back","average":{"1min":1.1060000000000001,"5min":1.1060000000000001,"15min":1.1060000000000001},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":2.4900000000000002,"5min":2.4900000000000002,"15min":2.4900000000000002},"last":1.4710000000000001},{"interface":"front","average":{"1min":1.3120000000000001,"5min":1.3120000000000001,"15min":1.3120000000000001},"min":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":3.2810000000000001,"5min":3.2810000000000001,"15min":3.2810000000000001},"last":1.5109999999999999}]},{"osd":8,"last update":"Wed Sep 18 10:33:17 2024","interfaces":[{"interface":"back","average":{"1min":1.258,"5min":1.258,"15min":1.258},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":4.4969999999999999,"5min":4.4969999999999999,"15min":4.4969999999999999},"last":1.4850000000000001},{"interface":"front","average":{"1min":1.1419999999999999,"5min":1.1419999999999999,"15min":1.1419999999999999},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":2.1629999999999998,"5min":2.1629999999999998,"15min":2.1629999999999998},"last":1.569}]},{"osd":9,"last update":"Wed Sep 18 10:33:17 2024","interfaces":[{"interface":"back","average":{"1min":1.4419999999999999,"5min":1.4419999999999999,"15min":1.4419999999999999},"min":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"max":{"1min":3.2160000000000002,"5min":3.2160000000000002,"15min":3.2160000000000002},"last":1.601},{"interface":"front","average":{"1min":1.2410000000000001,"5min":1.2410000000000001,"15min":1.2410000000000001},"min":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"max":{"1min":1.9430000000000001,"5min":1.9430000000000001,"15min":1.9430000000000001},"last":1.619}]}]},{"osd":0,"up_from":0,"seq":24,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":298000,"kb_used_data":636,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":104559600,"statfs":{"total":107374182400,"available":107069030400,"internally_reserved":0,"allocated":651264,"data_stored":308977,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[2,3,4,5,6,7,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Sep 18 10:33:15 2024","interfaces":[{"interface":"back","average":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":1.5880000000000001,"5min":1.5880000000000001,"15min":1.5880000000000001},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.917,"5min":1.917,"15min":1.917},"last":1.0600000000000001}]},{"osd":2,"last update":"Wed Sep 18 10:33:15 2024","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.746,"15min":0.746},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.454,"5min":1.454,"15min":1.454},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"min":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.222,"5min":1.222,"15min":1.222},"last":0.81699999999999995}]},{"osd":3,"last update":"Wed Sep 18 10:33:15 2024","interfaces":[{"interface":"back","average":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":1.5509999999999999,"5min":1.5509999999999999,"15min":1.5509999999999999},"last":1.036},{"interface":"front","average":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.6339999999999999,"5min":1.6339999999999999,"15min":1.6339999999999999},"last":0.80200000000000005}]},{"osd":4,"last update":"Wed Sep 18 10:33:15 2024","interfaces":[{"interface":"back","average":{"1min":1,"5min":1,"15min":1},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":1.6339999999999999,"5min":1.6339999999999999,"15min":1.6339999999999999},"last":0.98999999999999999},{"interface":"front","average":{"1min":1.093,"5min":1.093,"15min":1.093},"min":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.704,"5min":1.704,"15min":1.704},"last":1.135}]},{"osd":5,"last update":"Wed Sep 18 10:33:15 2024","interfaces":[{"interface":"back","average":{"1min":1.1699999999999999,"5min":1.1699999999999999,"15min":1.1699999999999999},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.9590000000000001,"5min":1.9590000000000001,"15min":1.9590000000000001},"last":1.1599999999999999},{"interface":"front","average":{"1min":1.2050000000000001,"5min":1.2050000000000001,"15min":1.2050000000000001},"min":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"max":{"1min":2.206,"5min":2.206,"15min":2.206},"last":1.0800000000000001}]},{"osd":6,"last update":"Wed Sep 18 10:33:15 2024","interfaces":[{"interface":"back","average":{"1min":1.157,"5min":1.157,"15min":1.157},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":1.74,"5min":1.74,"15min":1.74},"last":1.4159999999999999},{"interface":"front","average":{"1min":1.135,"5min":1.135,"15min":1.135},"min":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"max":{"1min":2.8140000000000001,"5min":2.8140000000000001,"15min":2.8140000000000001},"last":1.3919999999999999}]},{"osd":7,"last update":"Wed Sep 18 10:33:15 2024","interfaces":[{"interface":"back","average":{"1min":1.268,"5min":1.268,"15min":1.268},"min":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"max":{"1min":2.004,"5min":2.004,"15min":2.004},"last":2.6909999999999998},{"interface":"front","average":{"1min":1.357,"5min":1.357,"15min":1.357},"min":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"max":{"1min":2.7250000000000001,"5min":2.7250000000000001,"15min":2.7250000000000001},"last":1.8899999999999999}]},{"osd":8,"last update":"Wed Sep 18 10:33:15 2024","interfaces":[{"interface":"back","average":{"1min":1.3049999999999999,"5min":1.3049999999999999,"15min":1.3049999999999999},"min":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":2.1419999999999999,"5min":2.1419999999999999,"15min":2.1419999999999999},"last":2.5499999999999998},{"interface":"front","average":{"1min":1.3779999999999999,"5min":1.3779999999999999,"15min":1.3779999999999999},"min":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"max":{"1min":2.8980000000000001,"5min":2.8980000000000001,"15min":2.8980000000000001},"last":1.6279999999999999}]},{"osd":9,"last update":"Wed Sep 18 10:33:15 2024","interfaces":[{"interface":"back","average":{"1min":1.2989999999999999,"5min":1.2989999999999999,"15min":1.2989999999999999},"min":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"max":{"1min":2.3109999999999999,"5min":2.3109999999999999,"15min":2.3109999999999999},"last":1.8140000000000001},{"interface":"front","average":{"1min":1.333,"5min":1.333,"15min":1.333},"min":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":2.2639999999999998,"5min":2.2639999999999998,"15min":2.2639999999999998},"last":2.7719999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-18T10:34:04.247 INFO:tasks.ceph.ceph_manager.ceph:PG 1.5 is not active+clean 2024-09-18T10:34:04.247 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '1.5', 'version': "0'0", 'reported_seq': 85, 'reported_epoch': 57, 'state': 'active+undersized', 'last_fresh': '2024-09-18T10:34:00.271592+0000', 'last_change': '2024-09-18T10:34:00.271514+0000', 'last_active': '2024-09-18T10:34:00.271592+0000', 'last_peered': '2024-09-18T10:34:00.271592+0000', 'last_clean': '2024-09-18T10:33:56.981057+0000', 'last_became_active': '2024-09-18T10:33:59.102191+0000', 'last_became_peered': '2024-09-18T10:33:59.102191+0000', 'last_unstale': '2024-09-18T10:34:00.271592+0000', 'last_undegraded': '2024-09-18T10:34:00.271592+0000', 'last_fullsized': '2024-09-18T10:33:59.101046+0000', 'mapping_epoch': 55, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 49, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-18T10:32:19.443095+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-18T10:32:19.443095+0000', 'last_clean_scrub_stamp': '2024-09-18T10:32:19.443095+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-09-19T13:05:55.380579+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.0004848, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [9], 'acting': [9], 'avail_no_missing': ['9'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 9, 'acting_primary': 9, 'purged_snaps': []} 2024-09-18T10:34:04.248 INFO:tasks.ceph.ceph_manager.ceph:PG 1.4 is not active+clean 2024-09-18T10:34:04.248 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '1.4', 'version': "0'0", 'reported_seq': 45, 'reported_epoch': 56, 'state': 'active+undersized+wait', 'last_fresh': '2024-09-18T10:33:59.102960+0000', 'last_change': '2024-09-18T10:33:59.102253+0000', 'last_active': '2024-09-18T10:33:59.102960+0000', 'last_peered': '2024-09-18T10:33:59.102960+0000', 'last_clean': '2024-09-18T10:33:47.951434+0000', 'last_became_active': '2024-09-18T10:33:59.102253+0000', 'last_became_peered': '2024-09-18T10:33:59.102253+0000', 'last_unstale': '2024-09-18T10:33:59.102960+0000', 'last_undegraded': '2024-09-18T10:33:59.102960+0000', 'last_fullsized': '2024-09-18T10:33:59.100957+0000', 'mapping_epoch': 55, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 49, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-18T10:32:19.443095+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-18T10:32:19.443095+0000', 'last_clean_scrub_stamp': '2024-09-18T10:32:19.443095+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-09-19T19:47:09.963588+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6], 'acting': [6], 'avail_no_missing': ['6'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-18T10:34:04.248 INFO:tasks.ceph.ceph_manager.ceph:PG 1.3 is not active+clean 2024-09-18T10:34:04.248 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '1.3', 'version': "18'1", 'reported_seq': 70, 'reported_epoch': 57, 'state': 'active+undersized+degraded+wait', 'last_fresh': '2024-09-18T10:34:00.106984+0000', 'last_change': '2024-09-18T10:34:00.106902+0000', 'last_active': '2024-09-18T10:34:00.106984+0000', 'last_peered': '2024-09-18T10:34:00.106984+0000', 'last_clean': '2024-09-18T10:33:59.061965+0000', 'last_became_active': '2024-09-18T10:34:00.106902+0000', 'last_became_peered': '2024-09-18T10:34:00.106902+0000', 'last_unstale': '2024-09-18T10:34:00.106984+0000', 'last_undegraded': '2024-09-18T10:34:00.106209+0000', 'last_fullsized': '2024-09-18T10:34:00.106121+0000', 'mapping_epoch': 55, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 49, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-18T10:32:19.443095+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-18T10:32:19.443095+0000', 'last_clean_scrub_stamp': '2024-09-18T10:32:19.443095+0000', 'objects_scrubbed': 0, 'log_size': 1, 'log_dups_size': 0, 'ondisk_log_size': 1, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-09-19T21:39:17.714747+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 1, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3], 'acting': [3], 'avail_no_missing': ['3'], 'object_location_counts': [{'shards': '3', 'objects': 1}], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-18T10:34:04.248 INFO:tasks.ceph.ceph_manager.ceph:PG 1.2 is not active+clean 2024-09-18T10:34:04.248 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '1.2', 'version': "20'2", 'reported_seq': 73, 'reported_epoch': 56, 'state': 'active+undersized+degraded+wait', 'last_fresh': '2024-09-18T10:33:59.102609+0000', 'last_change': '2024-09-18T10:33:59.102002+0000', 'last_active': '2024-09-18T10:33:59.102609+0000', 'last_peered': '2024-09-18T10:33:59.102609+0000', 'last_clean': '2024-09-18T10:33:58.228743+0000', 'last_became_active': '2024-09-18T10:33:59.102002+0000', 'last_became_peered': '2024-09-18T10:33:59.102002+0000', 'last_unstale': '2024-09-18T10:33:59.102609+0000', 'last_undegraded': '2024-09-18T10:33:59.100769+0000', 'last_fullsized': '2024-09-18T10:33:59.100508+0000', 'mapping_epoch': 55, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 49, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-18T10:32:19.443095+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-18T10:32:19.443095+0000', 'last_clean_scrub_stamp': '2024-09-18T10:32:19.443095+0000', 'objects_scrubbed': 0, 'log_size': 2, 'log_dups_size': 0, 'ondisk_log_size': 2, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2024-09-19T19:07:49.365548+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0.001947504, 'stat_sum': {'num_bytes': 19, 'num_objects': 1, 'num_object_clones': 0, 'num_object_copies': 2, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 1, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 1, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 2, 'num_write_kb': 2, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 1, 'num_bytes_recovered': 19, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4], 'acting': [4], 'avail_no_missing': ['4'], 'object_location_counts': [{'shards': '4', 'objects': 1}], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-18T10:34:04.248 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-18T10:34:04.590 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-18T10:34:04.590 INFO:teuthology.orchestra.run.smithi134.stdout:{"fsid":"909a0a21-79ad-49e6-bd09-2f76d1ea12ab","health":{"status":"HEALTH_WARN","checks":{"OSDMAP_FLAGS":{"severity":"HEALTH_WARN","summary":{"message":"noout flag(s) set","count":5},"muted":false},"PG_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"Degraded data redundancy: 2/4 objects degraded (50.000%), 2 pgs degraded","count":2},"muted":false}},"mutes":[]},"election_epoch":3,"quorum":[0],"quorum_names":["a"],"quorum_age":117,"monmap":{"epoch":1,"min_mon_release_name":"quincy","num_mons":1},"osdmap":{"epoch":60,"num_osds":10,"num_up_osds":10,"osd_up_since":1726655642,"num_in_osds":10,"osd_in_since":1726655529,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":4},{"state_name":"active+undersized+degraded+wait","count":2},{"state_name":"active+undersized+wait","count":1},{"state_name":"active+undersized","count":1}],"num_pgs":8,"num_pools":1,"num_objects":2,"data_bytes":19,"bytes_used":3051429888,"bytes_avail":1070690394112,"bytes_total":1073741824000,"degraded_objects":2,"degraded_total":4,"degraded_ratio":0.5},"fsmap":{"epoch":1,"by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":3,"modified":"2024-09-18T10:33:17.420416+0000","services":{}},"progress_events":{}} 2024-09-18T10:34:04.604 INFO:tasks.ceph.ceph_manager.ceph:no progress seen, keeping timeout for now 2024-09-18T10:34:04.604 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-18T10:34:04.894 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-18T10:34:04.897 INFO:teuthology.orchestra.run.smithi134.stderr:dumped all 2024-09-18T10:34:04.909 INFO:teuthology.orchestra.run.smithi134.stdout:{"pg_ready":true,"pg_map":{"version":105,"stamp":"2024-09-18T10:34:03.431290+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":12,"acting":12,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":10,"num_per_pool_osds":10,"num_per_pool_omap_osds":8,"kb":1048576000,"kb_used":2979912,"kb_used_data":6308,"kb_used_omap":16,"kb_used_meta":2973423,"kb_avail":1045596088,"statfs":{"total":1073741824000,"available":1070690394112,"internally_reserved":0,"allocated":6459392,"data_stored":3051408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":17040,"internal_metadata":3044785520},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000990"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":45,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-18T10:33:59.097752+0000","last_change":"2024-09-18T10:33:52.036028+0000","last_active":"2024-09-18T10:33:59.097752+0000","last_peered":"2024-09-18T10:33:59.097752+0000","last_clean":"2024-09-18T10:33:59.097752+0000","last_became_active":"2024-09-18T10:33:52.035591+0000","last_became_peered":"2024-09-18T10:33:52.035591+0000","last_unstale":"2024-09-18T10:33:59.097752+0000","last_undegraded":"2024-09-18T10:33:59.097752+0000","last_fullsized":"2024-09-18T10:33:59.097752+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T10:32:19.443095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:32:19.443095+0000","last_clean_scrub_stamp":"2024-09-18T10:32:19.443095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T18:06:23.838644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00038275899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":61,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-18T10:33:59.101190+0000","last_change":"2024-09-18T10:33:52.042125+0000","last_active":"2024-09-18T10:33:59.101190+0000","last_peered":"2024-09-18T10:33:59.101190+0000","last_clean":"2024-09-18T10:33:59.101190+0000","last_became_active":"2024-09-18T10:33:52.041895+0000","last_became_peered":"2024-09-18T10:33:52.041895+0000","last_unstale":"2024-09-18T10:33:59.101190+0000","last_undegraded":"2024-09-18T10:33:59.101190+0000","last_fullsized":"2024-09-18T10:33:59.101190+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T10:32:19.443095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:32:19.443095+0000","last_clean_scrub_stamp":"2024-09-18T10:32:19.443095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T15:18:02.048451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034975400000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":85,"reported_epoch":57,"state":"active+undersized","last_fresh":"2024-09-18T10:34:00.271592+0000","last_change":"2024-09-18T10:34:00.271514+0000","last_active":"2024-09-18T10:34:00.271592+0000","last_peered":"2024-09-18T10:34:00.271592+0000","last_clean":"2024-09-18T10:33:56.981057+0000","last_became_active":"2024-09-18T10:33:59.102191+0000","last_became_peered":"2024-09-18T10:33:59.102191+0000","last_unstale":"2024-09-18T10:34:00.271592+0000","last_undegraded":"2024-09-18T10:34:00.271592+0000","last_fullsized":"2024-09-18T10:33:59.101046+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T10:32:19.443095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:32:19.443095+0000","last_clean_scrub_stamp":"2024-09-18T10:32:19.443095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T13:05:55.380579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048480000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9],"acting":[9],"avail_no_missing":["9"],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":45,"reported_epoch":56,"state":"active+undersized+wait","last_fresh":"2024-09-18T10:33:59.102960+0000","last_change":"2024-09-18T10:33:59.102253+0000","last_active":"2024-09-18T10:33:59.102960+0000","last_peered":"2024-09-18T10:33:59.102960+0000","last_clean":"2024-09-18T10:33:47.951434+0000","last_became_active":"2024-09-18T10:33:59.102253+0000","last_became_peered":"2024-09-18T10:33:59.102253+0000","last_unstale":"2024-09-18T10:33:59.102960+0000","last_undegraded":"2024-09-18T10:33:59.102960+0000","last_fullsized":"2024-09-18T10:33:59.100957+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T10:32:19.443095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:32:19.443095+0000","last_clean_scrub_stamp":"2024-09-18T10:32:19.443095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T19:47:09.963588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6],"acting":[6],"avail_no_missing":["6"],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.3","version":"18'1","reported_seq":70,"reported_epoch":57,"state":"active+undersized+degraded+wait","last_fresh":"2024-09-18T10:34:00.106984+0000","last_change":"2024-09-18T10:34:00.106902+0000","last_active":"2024-09-18T10:34:00.106984+0000","last_peered":"2024-09-18T10:34:00.106984+0000","last_clean":"2024-09-18T10:33:59.061965+0000","last_became_active":"2024-09-18T10:34:00.106902+0000","last_became_peered":"2024-09-18T10:34:00.106902+0000","last_unstale":"2024-09-18T10:34:00.106984+0000","last_undegraded":"2024-09-18T10:34:00.106209+0000","last_fullsized":"2024-09-18T10:34:00.106121+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T10:32:19.443095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:32:19.443095+0000","last_clean_scrub_stamp":"2024-09-18T10:32:19.443095+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T21:39:17.714747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3],"acting":[3],"avail_no_missing":["3"],"object_location_counts":[{"shards":"3","objects":1}],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.2","version":"20'2","reported_seq":73,"reported_epoch":56,"state":"active+undersized+degraded+wait","last_fresh":"2024-09-18T10:33:59.102609+0000","last_change":"2024-09-18T10:33:59.102002+0000","last_active":"2024-09-18T10:33:59.102609+0000","last_peered":"2024-09-18T10:33:59.102609+0000","last_clean":"2024-09-18T10:33:58.228743+0000","last_became_active":"2024-09-18T10:33:59.102002+0000","last_became_peered":"2024-09-18T10:33:59.102002+0000","last_unstale":"2024-09-18T10:33:59.102609+0000","last_undegraded":"2024-09-18T10:33:59.100769+0000","last_fullsized":"2024-09-18T10:33:59.100508+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T10:32:19.443095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:32:19.443095+0000","last_clean_scrub_stamp":"2024-09-18T10:32:19.443095+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T19:07:49.365548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019475040000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":1,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4],"acting":[4],"avail_no_missing":["4"],"object_location_counts":[{"shards":"4","objects":1}],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":42,"reported_epoch":55,"state":"active+clean","last_fresh":"2024-09-18T10:33:59.061961+0000","last_change":"2024-09-18T10:33:52.038532+0000","last_active":"2024-09-18T10:33:59.061961+0000","last_peered":"2024-09-18T10:33:59.061961+0000","last_clean":"2024-09-18T10:33:59.061961+0000","last_became_active":"2024-09-18T10:33:52.038239+0000","last_became_peered":"2024-09-18T10:33:52.038239+0000","last_unstale":"2024-09-18T10:33:59.061961+0000","last_undegraded":"2024-09-18T10:33:59.061961+0000","last_fullsized":"2024-09-18T10:33:59.061961+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T10:32:19.443095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:32:19.443095+0000","last_clean_scrub_stamp":"2024-09-18T10:32:19.443095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T18:34:12.717302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062722600000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":62,"reported_epoch":56,"state":"active+clean","last_fresh":"2024-09-18T10:33:59.400931+0000","last_change":"2024-09-18T10:33:52.037375+0000","last_active":"2024-09-18T10:33:59.400931+0000","last_peered":"2024-09-18T10:33:59.400931+0000","last_clean":"2024-09-18T10:33:59.400931+0000","last_became_active":"2024-09-18T10:33:52.037028+0000","last_became_peered":"2024-09-18T10:33:52.037028+0000","last_unstale":"2024-09-18T10:33:59.400931+0000","last_undegraded":"2024-09-18T10:33:59.400931+0000","last_fullsized":"2024-09-18T10:33:59.400931+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T10:32:19.443095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:32:19.443095+0000","last_clean_scrub_stamp":"2024-09-18T10:32:19.443095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T19:41:50.364436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9],"acting":[5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":2,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":12,"acting":12,"num_store_stats":9}],"osd_stats":[{"osd":9,"up_from":41,"seq":176093659159,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":298004,"kb_used_data":648,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":104559596,"statfs":{"total":107374182400,"available":107069026304,"internally_reserved":0,"allocated":663552,"data_stored":316651,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[2,3,4,5,6,7,8],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 10:33:18 2024","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":1.1930000000000001,"5min":1.1930000000000001,"15min":1.1930000000000001},"last":1.121},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.3029999999999999,"5min":1.3029999999999999,"15min":1.3029999999999999},"last":0.64900000000000002}]},{"osd":1,"last update":"Wed Sep 18 10:33:18 2024","interfaces":[{"interface":"back","average":{"1min":0.873,"5min":0.873,"15min":0.873},"min":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.3919999999999999,"5min":1.3919999999999999,"15min":1.3919999999999999},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":2.4359999999999999,"5min":2.4359999999999999,"15min":2.4359999999999999},"last":1.6379999999999999}]},{"osd":2,"last update":"Wed Sep 18 10:33:18 2024","interfaces":[{"interface":"back","average":{"1min":1.03,"5min":1.03,"15min":1.03},"min":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":1.526,"5min":1.526,"15min":1.526},"last":0.45300000000000001},{"interface":"front","average":{"1min":1.044,"5min":1.044,"15min":1.044},"min":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"max":{"1min":1.591,"5min":1.591,"15min":1.591},"last":0.52000000000000002}]},{"osd":3,"last update":"Wed Sep 18 10:33:18 2024","interfaces":[{"interface":"back","average":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.542,"5min":1.542,"15min":1.542},"last":0.63800000000000001},{"interface":"front","average":{"1min":1.274,"5min":1.274,"15min":1.274},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.8440000000000001,"5min":1.8440000000000001,"15min":1.8440000000000001},"last":0.98099999999999998}]},{"osd":4,"last update":"Wed Sep 18 10:33:18 2024","interfaces":[{"interface":"back","average":{"1min":1.2430000000000001,"5min":1.2430000000000001,"15min":1.2430000000000001},"min":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"max":{"1min":1.8180000000000001,"5min":1.8180000000000001,"15min":1.8180000000000001},"last":0.67300000000000004},{"interface":"front","average":{"1min":1.276,"5min":1.276,"15min":1.276},"min":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"max":{"1min":1.681,"5min":1.681,"15min":1.681},"last":0.96399999999999997}]},{"osd":5,"last update":"Wed Sep 18 10:33:18 2024","interfaces":[{"interface":"back","average":{"1min":1.3140000000000001,"5min":1.3140000000000001,"15min":1.3140000000000001},"min":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"max":{"1min":1.8600000000000001,"5min":1.8600000000000001,"15min":1.8600000000000001},"last":1.0269999999999999},{"interface":"front","average":{"1min":1.3009999999999999,"5min":1.3009999999999999,"15min":1.3009999999999999},"min":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"max":{"1min":1.7090000000000001,"5min":1.7090000000000001,"15min":1.7090000000000001},"last":0.78100000000000003}]},{"osd":6,"last update":"Wed Sep 18 10:33:18 2024","interfaces":[{"interface":"back","average":{"1min":1.3029999999999999,"5min":1.3029999999999999,"15min":1.3029999999999999},"min":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":2.778,"5min":2.778,"15min":2.778},"last":0.86599999999999999},{"interface":"front","average":{"1min":1.345,"5min":1.345,"15min":1.345},"min":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":1.921,"5min":1.921,"15min":1.921},"last":1.075}]},{"osd":7,"last update":"Wed Sep 18 10:33:18 2024","interfaces":[{"interface":"back","average":{"1min":1.3340000000000001,"5min":1.3340000000000001,"15min":1.3340000000000001},"min":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"max":{"1min":1.6830000000000001,"5min":1.6830000000000001,"15min":1.6830000000000001},"last":1.1439999999999999},{"interface":"front","average":{"1min":1.417,"5min":1.417,"15min":1.417},"min":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"max":{"1min":3.0390000000000001,"5min":3.0390000000000001,"15min":3.0390000000000001},"last":0.90600000000000003}]},{"osd":8,"last update":"Wed Sep 18 10:33:18 2024","interfaces":[{"interface":"back","average":{"1min":1.4330000000000001,"5min":1.4330000000000001,"15min":1.4330000000000001},"min":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"max":{"1min":2.8500000000000001,"5min":2.8500000000000001,"15min":2.8500000000000001},"last":0.93999999999999995},{"interface":"front","average":{"1min":1.4079999999999999,"5min":1.4079999999999999,"15min":1.4079999999999999},"min":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"max":{"1min":2.0070000000000001,"5min":2.0070000000000001,"15min":2.0070000000000001},"last":1.1910000000000001}]}]},{"osd":8,"up_from":48,"seq":206158430232,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":104857600,"kb_used":297956,"kb_used_data":600,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":104559644,"statfs":{"total":107374182400,"available":107069075456,"internally_reserved":0,"allocated":614400,"data_stored":285923,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,6,7,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 10:33:16 2024","interfaces":[{"interface":"back","average":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":1.26,"5min":1.26,"15min":1.26},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":1.238,"5min":1.238,"15min":1.238},"last":0.86299999999999999}]},{"osd":1,"last update":"Wed Sep 18 10:33:16 2024","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.751,"15min":0.751},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":1.129,"5min":1.129,"15min":1.129},"last":1.1359999999999999},{"interface":"front","average":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"min":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.2390000000000001,"5min":1.2390000000000001,"15min":1.2390000000000001},"last":0.92600000000000005}]},{"osd":2,"last update":"Wed Sep 18 10:33:16 2024","interfaces":[{"interface":"back","average":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":1.6439999999999999,"5min":1.6439999999999999,"15min":1.6439999999999999},"last":1.238},{"interface":"front","average":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":1.2529999999999999,"5min":1.2529999999999999,"15min":1.2529999999999999},"last":0.82299999999999995}]},{"osd":3,"last update":"Wed Sep 18 10:33:16 2024","interfaces":[{"interface":"back","average":{"1min":1.109,"5min":1.109,"15min":1.109},"min":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"max":{"1min":1.538,"5min":1.538,"15min":1.538},"last":1.458},{"interface":"front","average":{"1min":1.0389999999999999,"5min":1.0389999999999999,"15min":1.0389999999999999},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.4810000000000001,"5min":1.4810000000000001,"15min":1.4810000000000001},"last":1.581}]},{"osd":4,"last update":"Wed Sep 18 10:33:16 2024","interfaces":[{"interface":"back","average":{"1min":1.0620000000000001,"5min":1.0620000000000001,"15min":1.0620000000000001},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.8169999999999999,"5min":1.8169999999999999,"15min":1.8169999999999999},"last":1.917},{"interface":"front","average":{"1min":1.2549999999999999,"5min":1.2549999999999999,"15min":1.2549999999999999},"min":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"max":{"1min":2.1480000000000001,"5min":2.1480000000000001,"15min":2.1480000000000001},"last":1.3600000000000001}]},{"osd":5,"last update":"Wed Sep 18 10:33:16 2024","interfaces":[{"interface":"back","average":{"1min":1.216,"5min":1.216,"15min":1.216},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":2.3109999999999999,"5min":2.3109999999999999,"15min":2.3109999999999999},"last":1.637},{"interface":"front","average":{"1min":1.161,"5min":1.161,"15min":1.161},"min":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":2.0169999999999999,"5min":2.0169999999999999,"15min":2.0169999999999999},"last":1.7110000000000001}]},{"osd":6,"last update":"Wed Sep 18 10:33:16 2024","interfaces":[{"interface":"back","average":{"1min":1.2889999999999999,"5min":1.2889999999999999,"15min":1.2889999999999999},"min":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"max":{"1min":2.1899999999999999,"5min":2.1899999999999999,"15min":2.1899999999999999},"last":1.9470000000000001},{"interface":"front","average":{"1min":1.1799999999999999,"5min":1.1799999999999999,"15min":1.1799999999999999},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":2.069,"5min":2.069,"15min":2.069},"last":1.764}]},{"osd":7,"last update":"Wed Sep 18 10:33:16 2024","interfaces":[{"interface":"back","average":{"1min":1.2589999999999999,"5min":1.2589999999999999,"15min":1.2589999999999999},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":2.2130000000000001,"5min":2.2130000000000001,"15min":2.2130000000000001},"last":1.5129999999999999},{"interface":"front","average":{"1min":1.4039999999999999,"5min":1.4039999999999999,"15min":1.4039999999999999},"min":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"max":{"1min":2.2450000000000001,"5min":2.2450000000000001,"15min":2.2450000000000001},"last":2.0459999999999998}]},{"osd":9,"last update":"Wed Sep 18 10:33:16 2024","interfaces":[{"interface":"back","average":{"1min":1.304,"5min":1.304,"15min":1.304},"min":{"1min":0.63,"5min":0.63,"15min":0.63},"max":{"1min":2.1800000000000002,"5min":2.1800000000000002,"15min":2.1800000000000002},"last":2.0790000000000002},{"interface":"front","average":{"1min":1.3440000000000001,"5min":1.3440000000000001,"15min":1.3440000000000001},"min":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"max":{"1min":2.3969999999999998,"5min":2.3969999999999998,"15min":2.3969999999999998},"last":1.875}]}]},{"osd":7,"up_from":48,"seq":206158430231,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":297968,"kb_used_data":612,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":104559632,"statfs":{"total":107374182400,"available":107069063168,"internally_reserved":0,"allocated":626688,"data_stored":293647,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,6,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 10:33:17 2024","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.202,"5min":1.202,"15min":1.202},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.75,"5min":0.75,"15min":0.75},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":1.486,"5min":1.486,"15min":1.486},"last":0.98799999999999999}]},{"osd":1,"last update":"Wed Sep 18 10:33:17 2024","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.3009999999999999,"5min":1.3009999999999999,"15min":1.3009999999999999},"last":1.085},{"interface":"front","average":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.8480000000000001,"5min":1.8480000000000001,"15min":1.8480000000000001},"last":0.73799999999999999}]},{"osd":2,"last update":"Wed Sep 18 10:33:17 2024","interfaces":[{"interface":"back","average":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":1.5980000000000001,"5min":1.5980000000000001,"15min":1.5980000000000001},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.3440000000000001,"5min":1.3440000000000001,"15min":1.3440000000000001},"last":0.77800000000000002}]},{"osd":3,"last update":"Wed Sep 18 10:33:17 2024","interfaces":[{"interface":"back","average":{"1min":1.1000000000000001,"5min":1.1000000000000001,"15min":1.1000000000000001},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.9319999999999999,"5min":1.9319999999999999,"15min":1.9319999999999999},"last":0.85099999999999998},{"interface":"front","average":{"1min":1.0720000000000001,"5min":1.0720000000000001,"15min":1.0720000000000001},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.8879999999999999,"5min":1.8879999999999999,"15min":1.8879999999999999},"last":0.91400000000000003}]},{"osd":4,"last update":"Wed Sep 18 10:33:17 2024","interfaces":[{"interface":"back","average":{"1min":1.097,"5min":1.097,"15min":1.097},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":2.1379999999999999,"5min":2.1379999999999999,"15min":2.1379999999999999},"last":1.8680000000000001},{"interface":"front","average":{"1min":1.1060000000000001,"5min":1.1060000000000001,"15min":1.1060000000000001},"min":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":2.0110000000000001,"5min":2.0110000000000001,"15min":2.0110000000000001},"last":0.90400000000000003}]},{"osd":5,"last update":"Wed Sep 18 10:33:17 2024","interfaces":[{"interface":"back","average":{"1min":1.194,"5min":1.194,"15min":1.194},"min":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"max":{"1min":2.4889999999999999,"5min":2.4889999999999999,"15min":2.4889999999999999},"last":1.0329999999999999},{"interface":"front","average":{"1min":1.2230000000000001,"5min":1.2230000000000001,"15min":1.2230000000000001},"min":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"max":{"1min":2.4289999999999998,"5min":2.4289999999999998,"15min":2.4289999999999998},"last":2.056}]},{"osd":6,"last update":"Wed Sep 18 10:33:17 2024","interfaces":[{"interface":"back","average":{"1min":1.2769999999999999,"5min":1.2769999999999999,"15min":1.2769999999999999},"min":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"max":{"1min":2.0950000000000002,"5min":2.0950000000000002,"15min":2.0950000000000002},"last":1.341},{"interface":"front","average":{"1min":1.218,"5min":1.218,"15min":1.218},"min":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":1.9450000000000001,"5min":1.9450000000000001,"15min":1.9450000000000001},"last":1.1930000000000001}]},{"osd":8,"last update":"Wed Sep 18 10:33:17 2024","interfaces":[{"interface":"back","average":{"1min":1.2969999999999999,"5min":1.2969999999999999,"15min":1.2969999999999999},"min":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"max":{"1min":2.6389999999999998,"5min":2.6389999999999998,"15min":2.6389999999999998},"last":1.9810000000000001},{"interface":"front","average":{"1min":1.3600000000000001,"5min":1.3600000000000001,"15min":1.3600000000000001},"min":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"max":{"1min":2.1520000000000001,"5min":2.1520000000000001,"15min":2.1520000000000001},"last":1.2789999999999999}]},{"osd":9,"last update":"Wed Sep 18 10:33:17 2024","interfaces":[{"interface":"back","average":{"1min":1.2470000000000001,"5min":1.2470000000000001,"15min":1.2470000000000001},"min":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":2.0680000000000001,"5min":2.0680000000000001,"15min":2.0680000000000001},"last":2.2469999999999999},{"interface":"front","average":{"1min":1.282,"5min":1.282,"15min":1.282},"min":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"max":{"1min":1.8460000000000001,"5min":1.8460000000000001,"15min":1.8460000000000001},"last":2.9929999999999999}]}]},{"osd":6,"up_from":48,"seq":206158430231,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":297988,"kb_used_data":624,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":104559612,"statfs":{"total":107374182400,"available":107069042688,"internally_reserved":0,"allocated":638976,"data_stored":301295,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,7,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 10:33:17 2024","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":1.073,"5min":1.073,"15min":1.073},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":1.3720000000000001,"5min":1.3720000000000001,"15min":1.3720000000000001},"last":0.70499999999999996}]},{"osd":1,"last update":"Wed Sep 18 10:33:17 2024","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.751,"15min":0.751},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":1.6599999999999999,"5min":1.6599999999999999,"15min":1.6599999999999999},"last":1.095},{"interface":"front","average":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":2.0459999999999998,"5min":2.0459999999999998,"15min":2.0459999999999998},"last":0.61399999999999999}]},{"osd":2,"last update":"Wed Sep 18 10:33:17 2024","interfaces":[{"interface":"back","average":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":1.1399999999999999,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":0.77000000000000002},{"interface":"front","average":{"1min":1.052,"5min":1.052,"15min":1.052},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":1.907,"5min":1.907,"15min":1.907},"last":0.92400000000000004}]},{"osd":3,"last update":"Wed Sep 18 10:33:17 2024","interfaces":[{"interface":"back","average":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.7490000000000001,"5min":1.7490000000000001,"15min":1.7490000000000001},"last":0.749},{"interface":"front","average":{"1min":1.1830000000000001,"5min":1.1830000000000001,"15min":1.1830000000000001},"min":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"max":{"1min":1.8680000000000001,"5min":1.8680000000000001,"15min":1.8680000000000001},"last":1.1679999999999999}]},{"osd":4,"last update":"Wed Sep 18 10:33:17 2024","interfaces":[{"interface":"back","average":{"1min":1.2070000000000001,"5min":1.2070000000000001,"15min":1.2070000000000001},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.976,"5min":1.976,"15min":1.976},"last":1.25},{"interface":"front","average":{"1min":1.127,"5min":1.127,"15min":1.127},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":2.544,"5min":2.544,"15min":2.544},"last":1.6200000000000001}]},{"osd":5,"last update":"Wed Sep 18 10:33:17 2024","interfaces":[{"interface":"back","average":{"1min":1.2170000000000001,"5min":1.2170000000000001,"15min":1.2170000000000001},"min":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"max":{"1min":2.1539999999999999,"5min":2.1539999999999999,"15min":2.1539999999999999},"last":1.698},{"interface":"front","average":{"1min":1.296,"5min":1.296,"15min":1.296},"min":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":2.2469999999999999,"5min":2.2469999999999999,"15min":2.2469999999999999},"last":1.5649999999999999}]},{"osd":7,"last update":"Wed Sep 18 10:33:17 2024","interfaces":[{"interface":"back","average":{"1min":1.246,"5min":1.246,"15min":1.246},"min":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":2.6440000000000001,"5min":2.6440000000000001,"15min":2.6440000000000001},"last":1.2769999999999999},{"interface":"front","average":{"1min":1.7609999999999999,"5min":1.7609999999999999,"15min":1.7609999999999999},"min":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"max":{"1min":10.385,"5min":10.385,"15min":10.385},"last":1.633}]},{"osd":8,"last update":"Wed Sep 18 10:33:17 2024","interfaces":[{"interface":"back","average":{"1min":1.3640000000000001,"5min":1.3640000000000001,"15min":1.3640000000000001},"min":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"max":{"1min":2.444,"5min":2.444,"15min":2.444},"last":1.516},{"interface":"front","average":{"1min":1.274,"5min":1.274,"15min":1.274},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":2.714,"5min":2.714,"15min":2.714},"last":1.3580000000000001}]},{"osd":9,"last update":"Wed Sep 18 10:33:17 2024","interfaces":[{"interface":"back","average":{"1min":1.2789999999999999,"5min":1.2789999999999999,"15min":1.2789999999999999},"min":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"max":{"1min":2.2879999999999998,"5min":2.2879999999999998,"15min":2.2879999999999998},"last":1.768},{"interface":"front","average":{"1min":1.3240000000000001,"5min":1.3240000000000001,"15min":1.3240000000000001},"min":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":2.5960000000000001,"5min":2.5960000000000001,"15min":2.5960000000000001},"last":1.669}]}]},{"osd":5,"up_from":48,"seq":206158430233,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":298000,"kb_used_data":636,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":104559600,"statfs":{"total":107374182400,"available":107069030400,"internally_reserved":0,"allocated":651264,"data_stored":308977,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[2,3,4,6,7,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 10:33:18 2024","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.111,"5min":1.111,"15min":1.111},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"min":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.5249999999999999,"5min":1.5249999999999999,"15min":1.5249999999999999},"last":0.63}]},{"osd":1,"last update":"Wed Sep 18 10:33:18 2024","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.173,"5min":1.173,"15min":1.173},"last":0.79600000000000004},{"interface":"front","average":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.8280000000000001,"5min":1.8280000000000001,"15min":1.8280000000000001},"last":0.88500000000000001}]},{"osd":2,"last update":"Wed Sep 18 10:33:18 2024","interfaces":[{"interface":"back","average":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"min":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":1.4159999999999999,"5min":1.4159999999999999,"15min":1.4159999999999999},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"min":{"1min":0.627,"5min":0.627,"15min":0.627},"max":{"1min":2.3849999999999998,"5min":2.3849999999999998,"15min":2.3849999999999998},"last":0.63200000000000001}]},{"osd":3,"last update":"Wed Sep 18 10:33:18 2024","interfaces":[{"interface":"back","average":{"1min":1.079,"5min":1.079,"15min":1.079},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.923,"5min":1.923,"15min":1.923},"last":0.57099999999999995},{"interface":"front","average":{"1min":1.268,"5min":1.268,"15min":1.268},"min":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"max":{"1min":2.073,"5min":2.073,"15min":2.073},"last":0.72799999999999998}]},{"osd":4,"last update":"Wed Sep 18 10:33:18 2024","interfaces":[{"interface":"back","average":{"1min":1.123,"5min":1.123,"15min":1.123},"min":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"max":{"1min":2.141,"5min":2.141,"15min":2.141},"last":0.60699999999999998},{"interface":"front","average":{"1min":1.2490000000000001,"5min":1.2490000000000001,"15min":1.2490000000000001},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":2.3300000000000001,"5min":2.3300000000000001,"15min":2.3300000000000001},"last":1.05}]},{"osd":6,"last update":"Wed Sep 18 10:33:18 2024","interfaces":[{"interface":"back","average":{"1min":1.2769999999999999,"5min":1.2769999999999999,"15min":1.2769999999999999},"min":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"max":{"1min":1.806,"5min":1.806,"15min":1.806},"last":0.81499999999999995},{"interface":"front","average":{"1min":1.274,"5min":1.274,"15min":1.274},"min":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"max":{"1min":2.6120000000000001,"5min":2.6120000000000001,"15min":2.6120000000000001},"last":0.70199999999999996}]},{"osd":7,"last update":"Wed Sep 18 10:33:18 2024","interfaces":[{"interface":"back","average":{"1min":1.3380000000000001,"5min":1.3380000000000001,"15min":1.3380000000000001},"min":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"max":{"1min":2.661,"5min":2.661,"15min":2.661},"last":1.018},{"interface":"front","average":{"1min":1.3819999999999999,"5min":1.3819999999999999,"15min":1.3819999999999999},"min":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"max":{"1min":2.548,"5min":2.548,"15min":2.548},"last":0.89600000000000002}]},{"osd":8,"last update":"Wed Sep 18 10:33:18 2024","interfaces":[{"interface":"back","average":{"1min":1.359,"5min":1.359,"15min":1.359},"min":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"max":{"1min":2.7080000000000002,"5min":2.7080000000000002,"15min":2.7080000000000002},"last":0.96599999999999997},{"interface":"front","average":{"1min":1.327,"5min":1.327,"15min":1.327},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":2.4399999999999999,"5min":2.4399999999999999,"15min":2.4399999999999999},"last":1.169}]},{"osd":9,"last update":"Wed Sep 18 10:33:18 2024","interfaces":[{"interface":"back","average":{"1min":1.3859999999999999,"5min":1.3859999999999999,"15min":1.3859999999999999},"min":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"max":{"1min":2.2130000000000001,"5min":2.2130000000000001,"15min":2.2130000000000001},"last":0.92700000000000005},{"interface":"front","average":{"1min":1.4019999999999999,"5min":1.4019999999999999,"15min":1.4019999999999999},"min":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"max":{"1min":2.782,"5min":2.782,"15min":2.782},"last":1.0860000000000001}]}]},{"osd":4,"up_from":48,"seq":206158430231,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":297992,"kb_used_data":628,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":104559608,"statfs":{"total":107374182400,"available":107069038592,"internally_reserved":0,"allocated":643072,"data_stored":301314,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,5,6,7,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 10:33:16 2024","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":1.0960000000000001,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.745,"5min":0.745,"15min":0.745},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.171,"5min":1.171,"15min":1.171},"last":0.70799999999999996}]},{"osd":1,"last update":"Wed Sep 18 10:33:16 2024","interfaces":[{"interface":"back","average":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":1.1579999999999999,"5min":1.1579999999999999,"15min":1.1579999999999999},"last":1.0820000000000001},{"interface":"front","average":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.583,"5min":1.583,"15min":1.583},"last":0.77400000000000002}]},{"osd":2,"last update":"Wed Sep 18 10:33:16 2024","interfaces":[{"interface":"back","average":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":2.1139999999999999,"5min":2.1139999999999999,"15min":2.1139999999999999},"last":1.165},{"interface":"front","average":{"1min":0.995,"5min":0.995,"15min":0.995},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.5660000000000001,"5min":1.5660000000000001,"15min":1.5660000000000001},"last":0.56100000000000005}]},{"osd":3,"last update":"Wed Sep 18 10:33:16 2024","interfaces":[{"interface":"back","average":{"1min":1.0620000000000001,"5min":1.0620000000000001,"15min":1.0620000000000001},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.6200000000000001,"5min":1.6200000000000001,"15min":1.6200000000000001},"last":0.91400000000000003},{"interface":"front","average":{"1min":1.1699999999999999,"5min":1.1699999999999999,"15min":1.1699999999999999},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.732,"5min":1.732,"15min":1.732},"last":0.876}]},{"osd":5,"last update":"Wed Sep 18 10:33:16 2024","interfaces":[{"interface":"back","average":{"1min":1.159,"5min":1.159,"15min":1.159},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":2.2509999999999999,"5min":2.2509999999999999,"15min":2.2509999999999999},"last":1.0489999999999999},{"interface":"front","average":{"1min":1.1830000000000001,"5min":1.1830000000000001,"15min":1.1830000000000001},"min":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":1.873,"5min":1.873,"15min":1.873},"last":1.0049999999999999}]},{"osd":6,"last update":"Wed Sep 18 10:33:16 2024","interfaces":[{"interface":"back","average":{"1min":1.214,"5min":1.214,"15min":1.214},"min":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"max":{"1min":2.2879999999999998,"5min":2.2879999999999998,"15min":2.2879999999999998},"last":1.232},{"interface":"front","average":{"1min":1.25,"5min":1.25,"15min":1.25},"min":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":2.661,"5min":2.661,"15min":2.661},"last":1.137}]},{"osd":7,"last update":"Wed Sep 18 10:33:16 2024","interfaces":[{"interface":"back","average":{"1min":1.319,"5min":1.319,"15min":1.319},"min":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"max":{"1min":1.895,"5min":1.895,"15min":1.895},"last":1.415},{"interface":"front","average":{"1min":1.2450000000000001,"5min":1.2450000000000001,"15min":1.2450000000000001},"min":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":2.0579999999999998,"5min":2.0579999999999998,"15min":2.0579999999999998},"last":1.6619999999999999}]},{"osd":8,"last update":"Wed Sep 18 10:33:16 2024","interfaces":[{"interface":"back","average":{"1min":1.278,"5min":1.278,"15min":1.278},"min":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":2.173,"5min":2.173,"15min":2.173},"last":1.573},{"interface":"front","average":{"1min":1.3240000000000001,"5min":1.3240000000000001,"15min":1.3240000000000001},"min":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":1.9750000000000001,"5min":1.9750000000000001,"15min":1.9750000000000001},"last":1.375}]},{"osd":9,"last update":"Wed Sep 18 10:33:16 2024","interfaces":[{"interface":"back","average":{"1min":1.327,"5min":1.327,"15min":1.327},"min":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":2.448,"5min":2.448,"15min":2.448},"last":1.4419999999999999},{"interface":"front","average":{"1min":1.389,"5min":1.389,"15min":1.389},"min":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"max":{"1min":2.242,"5min":2.242,"15min":2.242},"last":1.474}]}]},{"osd":3,"up_from":48,"seq":206158430233,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":298000,"kb_used_data":636,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":104559600,"statfs":{"total":107374182400,"available":107069030400,"internally_reserved":0,"allocated":651264,"data_stored":308977,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[2,4,5,6,7,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 10:33:15 2024","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.095,"5min":1.095,"15min":1.095},"last":1.4199999999999999},{"interface":"front","average":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"min":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":2.121,"5min":2.121,"15min":2.121},"last":0.84499999999999997}]},{"osd":1,"last update":"Wed Sep 18 10:33:15 2024","interfaces":[{"interface":"back","average":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":1.9259999999999999,"5min":1.9259999999999999,"15min":1.9259999999999999},"last":1.1140000000000001},{"interface":"front","average":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"min":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":1.2090000000000001,"5min":1.2090000000000001,"15min":1.2090000000000001},"last":0.80800000000000005}]},{"osd":2,"last update":"Wed Sep 18 10:33:15 2024","interfaces":[{"interface":"back","average":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.204,"5min":1.204,"15min":1.204},"last":0.90900000000000003},{"interface":"front","average":{"1min":1.1870000000000001,"5min":1.1870000000000001,"15min":1.1870000000000001},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":3.2040000000000002,"5min":3.2040000000000002,"15min":3.2040000000000002},"last":2.6230000000000002}]},{"osd":4,"last update":"Wed Sep 18 10:33:15 2024","interfaces":[{"interface":"back","average":{"1min":1.071,"5min":1.071,"15min":1.071},"min":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":1.49,"5min":1.49,"15min":1.49},"last":0.82799999999999996},{"interface":"front","average":{"1min":1.266,"5min":1.266,"15min":1.266},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":2.7519999999999998,"5min":2.7519999999999998,"15min":2.7519999999999998},"last":1.016}]},{"osd":5,"last update":"Wed Sep 18 10:33:15 2024","interfaces":[{"interface":"back","average":{"1min":1.1830000000000001,"5min":1.1830000000000001,"15min":1.1830000000000001},"min":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"max":{"1min":2.0630000000000002,"5min":2.0630000000000002,"15min":2.0630000000000002},"last":0.71299999999999997},{"interface":"front","average":{"1min":1.1259999999999999,"5min":1.1259999999999999,"15min":1.1259999999999999},"min":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"max":{"1min":1.5149999999999999,"5min":1.5149999999999999,"15min":1.5149999999999999},"last":1.115}]},{"osd":6,"last update":"Wed Sep 18 10:33:15 2024","interfaces":[{"interface":"back","average":{"1min":1.3620000000000001,"5min":1.3620000000000001,"15min":1.3620000000000001},"min":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":4.1029999999999998,"5min":4.1029999999999998,"15min":4.1029999999999998},"last":0.97699999999999998},{"interface":"front","average":{"1min":1.268,"5min":1.268,"15min":1.268},"min":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"max":{"1min":2.2789999999999999,"5min":2.2789999999999999,"15min":2.2789999999999999},"last":1.427}]},{"osd":7,"last update":"Wed Sep 18 10:33:15 2024","interfaces":[{"interface":"back","average":{"1min":1.2010000000000001,"5min":1.2010000000000001,"15min":1.2010000000000001},"min":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"max":{"1min":2.1880000000000002,"5min":2.1880000000000002,"15min":2.1880000000000002},"last":2.7589999999999999},{"interface":"front","average":{"1min":1.282,"5min":1.282,"15min":1.282},"min":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"max":{"1min":3.0409999999999999,"5min":3.0409999999999999,"15min":3.0409999999999999},"last":1.2}]},{"osd":8,"last update":"Wed Sep 18 10:33:15 2024","interfaces":[{"interface":"back","average":{"1min":1.5109999999999999,"5min":1.5109999999999999,"15min":1.5109999999999999},"min":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"max":{"1min":3.5590000000000002,"5min":3.5590000000000002,"15min":3.5590000000000002},"last":1.3360000000000001},{"interface":"front","average":{"1min":1.3540000000000001,"5min":1.3540000000000001,"15min":1.3540000000000001},"min":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"max":{"1min":2.3919999999999999,"5min":2.3919999999999999,"15min":2.3919999999999999},"last":1.4850000000000001}]},{"osd":9,"last update":"Wed Sep 18 10:33:15 2024","interfaces":[{"interface":"back","average":{"1min":1.379,"5min":1.379,"15min":1.379},"min":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"max":{"1min":3.1259999999999999,"5min":3.1259999999999999,"15min":3.1259999999999999},"last":1.522},{"interface":"front","average":{"1min":1.5109999999999999,"5min":1.5109999999999999,"15min":1.5109999999999999},"min":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"max":{"1min":3.6339999999999999,"5min":3.6339999999999999,"15min":3.6339999999999999},"last":1.6020000000000001}]}]},{"osd":2,"up_from":48,"seq":206158430231,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":104857600,"kb_used":297996,"kb_used_data":636,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":104559604,"statfs":{"total":107374182400,"available":107069034496,"internally_reserved":0,"allocated":651264,"data_stored":308977,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,3,4,5,6,7,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 10:33:18 2024","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1.109,"5min":1.109,"15min":1.109},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":1.5029999999999999,"5min":1.5029999999999999,"15min":1.5029999999999999},"last":0.69299999999999995}]},{"osd":1,"last update":"Wed Sep 18 10:33:18 2024","interfaces":[{"interface":"back","average":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"min":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.651,"5min":1.651,"15min":1.651},"last":0.71799999999999997},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":1.1559999999999999,"5min":1.1559999999999999,"15min":1.1559999999999999},"last":1.1599999999999999}]},{"osd":3,"last update":"Wed Sep 18 10:33:18 2024","interfaces":[{"interface":"back","average":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":1.2869999999999999,"5min":1.2869999999999999,"15min":1.2869999999999999},"last":0.92800000000000005},{"interface":"front","average":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.7390000000000001,"5min":1.7390000000000001,"15min":1.7390000000000001},"last":0.83899999999999997}]},{"osd":4,"last update":"Wed Sep 18 10:33:18 2024","interfaces":[{"interface":"back","average":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"min":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":1.333,"5min":1.333,"15min":1.333},"last":1.329},{"interface":"front","average":{"1min":1.151,"5min":1.151,"15min":1.151},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.7869999999999999,"5min":1.7869999999999999,"15min":1.7869999999999999},"last":0.89300000000000002}]},{"osd":5,"last update":"Wed Sep 18 10:33:18 2024","interfaces":[{"interface":"back","average":{"1min":1.1100000000000001,"5min":1.1100000000000001,"15min":1.1100000000000001},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.597,"5min":1.597,"15min":1.597},"last":1.2909999999999999},{"interface":"front","average":{"1min":1.246,"5min":1.246,"15min":1.246},"min":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"max":{"1min":1.873,"5min":1.873,"15min":1.873},"last":1.0029999999999999}]},{"osd":6,"last update":"Wed Sep 18 10:33:18 2024","interfaces":[{"interface":"back","average":{"1min":1.0649999999999999,"5min":1.0649999999999999,"15min":1.0649999999999999},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":1.696,"5min":1.696,"15min":1.696},"last":1.6259999999999999},{"interface":"front","average":{"1min":1.1479999999999999,"5min":1.1479999999999999,"15min":1.1479999999999999},"min":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"max":{"1min":1.5289999999999999,"5min":1.5289999999999999,"15min":1.5289999999999999},"last":1.605}]},{"osd":7,"last update":"Wed Sep 18 10:33:18 2024","interfaces":[{"interface":"back","average":{"1min":1.268,"5min":1.268,"15min":1.268},"min":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":1.9350000000000001,"5min":1.9350000000000001,"15min":1.9350000000000001},"last":1.492},{"interface":"front","average":{"1min":1.244,"5min":1.244,"15min":1.244},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":2.6419999999999999,"5min":2.6419999999999999,"15min":2.6419999999999999},"last":1.534}]},{"osd":8,"last update":"Wed Sep 18 10:33:18 2024","interfaces":[{"interface":"back","average":{"1min":1.306,"5min":1.306,"15min":1.306},"min":{"1min":1,"5min":1,"15min":1},"max":{"1min":1.613,"5min":1.613,"15min":1.613},"last":1.093},{"interface":"front","average":{"1min":1.278,"5min":1.278,"15min":1.278},"min":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":1.9830000000000001,"5min":1.9830000000000001,"15min":1.9830000000000001},"last":1.7}]},{"osd":9,"last update":"Wed Sep 18 10:33:18 2024","interfaces":[{"interface":"back","average":{"1min":1.2210000000000001,"5min":1.2210000000000001,"15min":1.2210000000000001},"min":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"max":{"1min":2.4220000000000002,"5min":2.4220000000000002,"15min":2.4220000000000002},"last":1.5720000000000001},{"interface":"front","average":{"1min":1.337,"5min":1.337,"15min":1.337},"min":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"max":{"1min":2.1349999999999998,"5min":2.1349999999999998,"15min":2.1349999999999998},"last":1.0609999999999999}]}]},{"osd":1,"up_from":0,"seq":24,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":298008,"kb_used_data":652,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":104559592,"statfs":{"total":107374182400,"available":107069022208,"internally_reserved":0,"allocated":667648,"data_stored":316670,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[2,3,4,5,6,7,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 10:33:17 2024","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":1.0980000000000001,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.89900000000000002},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"min":{"1min":0.248,"5min":0.248,"15min":0.248},"max":{"1min":1.3,"5min":1.3,"15min":1.3},"last":0.58699999999999997}]},{"osd":2,"last update":"Wed Sep 18 10:33:17 2024","interfaces":[{"interface":"back","average":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.206,"5min":1.206,"15min":1.206},"last":1.1020000000000001},{"interface":"front","average":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":1.54,"5min":1.54,"15min":1.54},"last":0.83199999999999996}]},{"osd":3,"last update":"Wed Sep 18 10:33:17 2024","interfaces":[{"interface":"back","average":{"1min":1.129,"5min":1.129,"15min":1.129},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":5.7569999999999997,"5min":5.7569999999999997,"15min":5.7569999999999997},"last":1.139},{"interface":"front","average":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":2.9830000000000001,"5min":2.9830000000000001,"15min":2.9830000000000001},"last":0.999}]},{"osd":4,"last update":"Wed Sep 18 10:33:17 2024","interfaces":[{"interface":"back","average":{"1min":1.1000000000000001,"5min":1.1000000000000001,"15min":1.1000000000000001},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":3.8700000000000001,"5min":3.8700000000000001,"15min":3.8700000000000001},"last":1.1899999999999999},{"interface":"front","average":{"1min":1.1359999999999999,"5min":1.1359999999999999,"15min":1.1359999999999999},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":2.6520000000000001,"5min":2.6520000000000001,"15min":2.6520000000000001},"last":1.296}]},{"osd":5,"last update":"Wed Sep 18 10:33:17 2024","interfaces":[{"interface":"back","average":{"1min":1.1160000000000001,"5min":1.1160000000000001,"15min":1.1160000000000001},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":2.0470000000000002,"5min":2.0470000000000002,"15min":2.0470000000000002},"last":1.21},{"interface":"front","average":{"1min":1.0920000000000001,"5min":1.0920000000000001,"15min":1.0920000000000001},"min":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"max":{"1min":1.6679999999999999,"5min":1.6679999999999999,"15min":1.6679999999999999},"last":1.4059999999999999}]},{"osd":6,"last update":"Wed Sep 18 10:33:17 2024","interfaces":[{"interface":"back","average":{"1min":1.3280000000000001,"5min":1.3280000000000001,"15min":1.3280000000000001},"min":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":3.1299999999999999,"5min":3.1299999999999999,"15min":3.1299999999999999},"last":1.5489999999999999},{"interface":"front","average":{"1min":1.123,"5min":1.123,"15min":1.123},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.9390000000000001,"5min":1.9390000000000001,"15min":1.9390000000000001},"last":1.3939999999999999}]},{"osd":7,"last update":"Wed Sep 18 10:33:17 2024","interfaces":[{"interface":"back","average":{"1min":1.1060000000000001,"5min":1.1060000000000001,"15min":1.1060000000000001},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":2.4900000000000002,"5min":2.4900000000000002,"15min":2.4900000000000002},"last":1.4710000000000001},{"interface":"front","average":{"1min":1.3120000000000001,"5min":1.3120000000000001,"15min":1.3120000000000001},"min":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":3.2810000000000001,"5min":3.2810000000000001,"15min":3.2810000000000001},"last":1.5109999999999999}]},{"osd":8,"last update":"Wed Sep 18 10:33:17 2024","interfaces":[{"interface":"back","average":{"1min":1.258,"5min":1.258,"15min":1.258},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":4.4969999999999999,"5min":4.4969999999999999,"15min":4.4969999999999999},"last":1.4850000000000001},{"interface":"front","average":{"1min":1.1419999999999999,"5min":1.1419999999999999,"15min":1.1419999999999999},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":2.1629999999999998,"5min":2.1629999999999998,"15min":2.1629999999999998},"last":1.569}]},{"osd":9,"last update":"Wed Sep 18 10:33:17 2024","interfaces":[{"interface":"back","average":{"1min":1.4419999999999999,"5min":1.4419999999999999,"15min":1.4419999999999999},"min":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"max":{"1min":3.2160000000000002,"5min":3.2160000000000002,"15min":3.2160000000000002},"last":1.601},{"interface":"front","average":{"1min":1.2410000000000001,"5min":1.2410000000000001,"15min":1.2410000000000001},"min":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"max":{"1min":1.9430000000000001,"5min":1.9430000000000001,"15min":1.9430000000000001},"last":1.619}]}]},{"osd":0,"up_from":0,"seq":24,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":298000,"kb_used_data":636,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":104559600,"statfs":{"total":107374182400,"available":107069030400,"internally_reserved":0,"allocated":651264,"data_stored":308977,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[2,3,4,5,6,7,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Sep 18 10:33:15 2024","interfaces":[{"interface":"back","average":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":1.5880000000000001,"5min":1.5880000000000001,"15min":1.5880000000000001},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.917,"5min":1.917,"15min":1.917},"last":1.0600000000000001}]},{"osd":2,"last update":"Wed Sep 18 10:33:15 2024","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.746,"15min":0.746},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.454,"5min":1.454,"15min":1.454},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"min":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.222,"5min":1.222,"15min":1.222},"last":0.81699999999999995}]},{"osd":3,"last update":"Wed Sep 18 10:33:15 2024","interfaces":[{"interface":"back","average":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":1.5509999999999999,"5min":1.5509999999999999,"15min":1.5509999999999999},"last":1.036},{"interface":"front","average":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.6339999999999999,"5min":1.6339999999999999,"15min":1.6339999999999999},"last":0.80200000000000005}]},{"osd":4,"last update":"Wed Sep 18 10:33:15 2024","interfaces":[{"interface":"back","average":{"1min":1,"5min":1,"15min":1},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":1.6339999999999999,"5min":1.6339999999999999,"15min":1.6339999999999999},"last":0.98999999999999999},{"interface":"front","average":{"1min":1.093,"5min":1.093,"15min":1.093},"min":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.704,"5min":1.704,"15min":1.704},"last":1.135}]},{"osd":5,"last update":"Wed Sep 18 10:33:15 2024","interfaces":[{"interface":"back","average":{"1min":1.1699999999999999,"5min":1.1699999999999999,"15min":1.1699999999999999},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.9590000000000001,"5min":1.9590000000000001,"15min":1.9590000000000001},"last":1.1599999999999999},{"interface":"front","average":{"1min":1.2050000000000001,"5min":1.2050000000000001,"15min":1.2050000000000001},"min":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"max":{"1min":2.206,"5min":2.206,"15min":2.206},"last":1.0800000000000001}]},{"osd":6,"last update":"Wed Sep 18 10:33:15 2024","interfaces":[{"interface":"back","average":{"1min":1.157,"5min":1.157,"15min":1.157},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":1.74,"5min":1.74,"15min":1.74},"last":1.4159999999999999},{"interface":"front","average":{"1min":1.135,"5min":1.135,"15min":1.135},"min":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"max":{"1min":2.8140000000000001,"5min":2.8140000000000001,"15min":2.8140000000000001},"last":1.3919999999999999}]},{"osd":7,"last update":"Wed Sep 18 10:33:15 2024","interfaces":[{"interface":"back","average":{"1min":1.268,"5min":1.268,"15min":1.268},"min":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"max":{"1min":2.004,"5min":2.004,"15min":2.004},"last":2.6909999999999998},{"interface":"front","average":{"1min":1.357,"5min":1.357,"15min":1.357},"min":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"max":{"1min":2.7250000000000001,"5min":2.7250000000000001,"15min":2.7250000000000001},"last":1.8899999999999999}]},{"osd":8,"last update":"Wed Sep 18 10:33:15 2024","interfaces":[{"interface":"back","average":{"1min":1.3049999999999999,"5min":1.3049999999999999,"15min":1.3049999999999999},"min":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":2.1419999999999999,"5min":2.1419999999999999,"15min":2.1419999999999999},"last":2.5499999999999998},{"interface":"front","average":{"1min":1.3779999999999999,"5min":1.3779999999999999,"15min":1.3779999999999999},"min":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"max":{"1min":2.8980000000000001,"5min":2.8980000000000001,"15min":2.8980000000000001},"last":1.6279999999999999}]},{"osd":9,"last update":"Wed Sep 18 10:33:15 2024","interfaces":[{"interface":"back","average":{"1min":1.2989999999999999,"5min":1.2989999999999999,"15min":1.2989999999999999},"min":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"max":{"1min":2.3109999999999999,"5min":2.3109999999999999,"15min":2.3109999999999999},"last":1.8140000000000001},{"interface":"front","average":{"1min":1.333,"5min":1.333,"15min":1.333},"min":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":2.2639999999999998,"5min":2.2639999999999998,"15min":2.2639999999999998},"last":2.7719999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-18T10:34:07.912 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-18T10:34:08.216 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-18T10:34:08.219 INFO:teuthology.orchestra.run.smithi134.stderr:dumped all 2024-09-18T10:34:08.231 INFO:teuthology.orchestra.run.smithi134.stdout:{"pg_ready":true,"pg_map":{"version":107,"stamp":"2024-09-18T10:34:07.432291+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":10,"num_per_pool_osds":10,"num_per_pool_omap_osds":8,"kb":1048576000,"kb_used":2980332,"kb_used_data":6728,"kb_used_omap":16,"kb_used_meta":2973423,"kb_avail":1045595668,"statfs":{"total":1073741824000,"available":1070689964032,"internally_reserved":0,"allocated":6889472,"data_stored":3327748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":17040,"internal_metadata":3044785520},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.324608"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":47,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-18T10:34:02.428277+0000","last_change":"2024-09-18T10:33:52.036028+0000","last_active":"2024-09-18T10:34:02.428277+0000","last_peered":"2024-09-18T10:34:02.428277+0000","last_clean":"2024-09-18T10:34:02.428277+0000","last_became_active":"2024-09-18T10:33:52.035591+0000","last_became_peered":"2024-09-18T10:33:52.035591+0000","last_unstale":"2024-09-18T10:34:02.428277+0000","last_undegraded":"2024-09-18T10:34:02.428277+0000","last_fullsized":"2024-09-18T10:34:02.428277+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T10:32:19.443095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:32:19.443095+0000","last_clean_scrub_stamp":"2024-09-18T10:32:19.443095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T18:06:23.838644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00038275899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":63,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-18T10:34:02.778343+0000","last_change":"2024-09-18T10:33:52.042125+0000","last_active":"2024-09-18T10:34:02.778343+0000","last_peered":"2024-09-18T10:34:02.778343+0000","last_clean":"2024-09-18T10:34:02.778343+0000","last_became_active":"2024-09-18T10:33:52.041895+0000","last_became_peered":"2024-09-18T10:33:52.041895+0000","last_unstale":"2024-09-18T10:34:02.778343+0000","last_undegraded":"2024-09-18T10:34:02.778343+0000","last_fullsized":"2024-09-18T10:34:02.778343+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T10:32:19.443095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:32:19.443095+0000","last_clean_scrub_stamp":"2024-09-18T10:32:19.443095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T15:18:02.048451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034975400000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":96,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-18T10:34:03.594986+0000","last_change":"2024-09-18T10:34:03.594986+0000","last_active":"2024-09-18T10:34:03.594986+0000","last_peered":"2024-09-18T10:34:03.594986+0000","last_clean":"2024-09-18T10:34:03.594986+0000","last_became_active":"2024-09-18T10:34:03.594428+0000","last_became_peered":"2024-09-18T10:34:03.594428+0000","last_unstale":"2024-09-18T10:34:03.594986+0000","last_undegraded":"2024-09-18T10:34:03.594986+0000","last_fullsized":"2024-09-18T10:34:03.594986+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T10:32:19.443095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:32:19.443095+0000","last_clean_scrub_stamp":"2024-09-18T10:32:19.443095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T18:25:37.752825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048480000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0],"acting":[9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":61,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-18T10:34:03.531946+0000","last_change":"2024-09-18T10:34:03.531946+0000","last_active":"2024-09-18T10:34:03.531946+0000","last_peered":"2024-09-18T10:34:03.531946+0000","last_clean":"2024-09-18T10:34:03.531946+0000","last_became_active":"2024-09-18T10:34:03.531651+0000","last_became_peered":"2024-09-18T10:34:03.531651+0000","last_unstale":"2024-09-18T10:34:03.531946+0000","last_undegraded":"2024-09-18T10:34:03.531946+0000","last_fullsized":"2024-09-18T10:34:03.531946+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T10:32:19.443095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:32:19.443095+0000","last_clean_scrub_stamp":"2024-09-18T10:32:19.443095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T19:47:09.963588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.3","version":"18'1","reported_seq":82,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-18T10:34:03.137967+0000","last_change":"2024-09-18T10:34:03.137967+0000","last_active":"2024-09-18T10:34:03.137967+0000","last_peered":"2024-09-18T10:34:03.137967+0000","last_clean":"2024-09-18T10:34:03.137967+0000","last_became_active":"2024-09-18T10:34:03.137466+0000","last_became_peered":"2024-09-18T10:34:03.137466+0000","last_unstale":"2024-09-18T10:34:03.137967+0000","last_undegraded":"2024-09-18T10:34:03.137967+0000","last_fullsized":"2024-09-18T10:34:03.137967+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T10:32:19.443095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:32:19.443095+0000","last_clean_scrub_stamp":"2024-09-18T10:32:19.443095+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T19:22:54.145211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.2","version":"20'2","reported_seq":85,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-18T10:34:03.136912+0000","last_change":"2024-09-18T10:34:03.136912+0000","last_active":"2024-09-18T10:34:03.136912+0000","last_peered":"2024-09-18T10:34:03.136912+0000","last_clean":"2024-09-18T10:34:03.136912+0000","last_became_active":"2024-09-18T10:34:03.136418+0000","last_became_peered":"2024-09-18T10:34:03.136418+0000","last_unstale":"2024-09-18T10:34:03.136912+0000","last_undegraded":"2024-09-18T10:34:03.136912+0000","last_fullsized":"2024-09-18T10:34:03.136912+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T10:32:19.443095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:32:19.443095+0000","last_clean_scrub_stamp":"2024-09-18T10:32:19.443095+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T21:39:17.714747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019475040000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":44,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-18T10:34:02.562266+0000","last_change":"2024-09-18T10:33:52.038532+0000","last_active":"2024-09-18T10:34:02.562266+0000","last_peered":"2024-09-18T10:34:02.562266+0000","last_clean":"2024-09-18T10:34:02.562266+0000","last_became_active":"2024-09-18T10:33:52.038239+0000","last_became_peered":"2024-09-18T10:33:52.038239+0000","last_unstale":"2024-09-18T10:34:02.562266+0000","last_undegraded":"2024-09-18T10:34:02.562266+0000","last_fullsized":"2024-09-18T10:34:02.562266+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T10:32:19.443095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:32:19.443095+0000","last_clean_scrub_stamp":"2024-09-18T10:32:19.443095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T18:34:12.717302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062722600000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":63,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-18T10:34:00.270430+0000","last_change":"2024-09-18T10:33:52.037375+0000","last_active":"2024-09-18T10:34:00.270430+0000","last_peered":"2024-09-18T10:34:00.270430+0000","last_clean":"2024-09-18T10:34:00.270430+0000","last_became_active":"2024-09-18T10:33:52.037028+0000","last_became_peered":"2024-09-18T10:33:52.037028+0000","last_unstale":"2024-09-18T10:34:00.270430+0000","last_undegraded":"2024-09-18T10:34:00.270430+0000","last_fullsized":"2024-09-18T10:34:00.270430+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T10:32:19.443095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:32:19.443095+0000","last_clean_scrub_stamp":"2024-09-18T10:32:19.443095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T19:41:50.364436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9],"acting":[5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":9}],"osd_stats":[{"osd":9,"up_from":41,"seq":176093659160,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":298036,"kb_used_data":672,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":104559564,"statfs":{"total":107374182400,"available":107068993536,"internally_reserved":0,"allocated":688128,"data_stored":332771,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,6,7,8],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 10:33:18 2024","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":1.1930000000000001,"5min":1.1930000000000001,"15min":1.1930000000000001},"last":0.91700000000000004},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.3029999999999999,"5min":1.3029999999999999,"15min":1.3029999999999999},"last":0.79600000000000004}]},{"osd":1,"last update":"Wed Sep 18 10:33:18 2024","interfaces":[{"interface":"back","average":{"1min":0.873,"5min":0.873,"15min":0.873},"min":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.3919999999999999,"5min":1.3919999999999999,"15min":1.3919999999999999},"last":0.83599999999999997},{"interface":"front","average":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":2.4359999999999999,"5min":2.4359999999999999,"15min":2.4359999999999999},"last":1.149}]},{"osd":2,"last update":"Wed Sep 18 10:33:18 2024","interfaces":[{"interface":"back","average":{"1min":1.03,"5min":1.03,"15min":1.03},"min":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":1.526,"5min":1.526,"15min":1.526},"last":0.69899999999999995},{"interface":"front","average":{"1min":1.044,"5min":1.044,"15min":1.044},"min":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"max":{"1min":1.591,"5min":1.591,"15min":1.591},"last":1.4590000000000001}]},{"osd":3,"last update":"Wed Sep 18 10:33:18 2024","interfaces":[{"interface":"back","average":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.542,"5min":1.542,"15min":1.542},"last":1.1830000000000001},{"interface":"front","average":{"1min":1.274,"5min":1.274,"15min":1.274},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.8440000000000001,"5min":1.8440000000000001,"15min":1.8440000000000001},"last":1.3979999999999999}]},{"osd":4,"last update":"Wed Sep 18 10:33:18 2024","interfaces":[{"interface":"back","average":{"1min":1.2430000000000001,"5min":1.2430000000000001,"15min":1.2430000000000001},"min":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"max":{"1min":1.8180000000000001,"5min":1.8180000000000001,"15min":1.8180000000000001},"last":1.04},{"interface":"front","average":{"1min":1.276,"5min":1.276,"15min":1.276},"min":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"max":{"1min":1.681,"5min":1.681,"15min":1.681},"last":1.2170000000000001}]},{"osd":5,"last update":"Wed Sep 18 10:33:18 2024","interfaces":[{"interface":"back","average":{"1min":1.3140000000000001,"5min":1.3140000000000001,"15min":1.3140000000000001},"min":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"max":{"1min":1.8600000000000001,"5min":1.8600000000000001,"15min":1.8600000000000001},"last":1},{"interface":"front","average":{"1min":1.3009999999999999,"5min":1.3009999999999999,"15min":1.3009999999999999},"min":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"max":{"1min":1.7090000000000001,"5min":1.7090000000000001,"15min":1.7090000000000001},"last":1.3360000000000001}]},{"osd":6,"last update":"Wed Sep 18 10:33:18 2024","interfaces":[{"interface":"back","average":{"1min":1.3029999999999999,"5min":1.3029999999999999,"15min":1.3029999999999999},"min":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":2.778,"5min":2.778,"15min":2.778},"last":1.0700000000000001},{"interface":"front","average":{"1min":1.345,"5min":1.345,"15min":1.345},"min":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":1.921,"5min":1.921,"15min":1.921},"last":1.522}]},{"osd":7,"last update":"Wed Sep 18 10:33:18 2024","interfaces":[{"interface":"back","average":{"1min":1.3340000000000001,"5min":1.3340000000000001,"15min":1.3340000000000001},"min":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"max":{"1min":1.6830000000000001,"5min":1.6830000000000001,"15min":1.6830000000000001},"last":1.595},{"interface":"front","average":{"1min":1.417,"5min":1.417,"15min":1.417},"min":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"max":{"1min":3.0390000000000001,"5min":3.0390000000000001,"15min":3.0390000000000001},"last":1.121}]},{"osd":8,"last update":"Wed Sep 18 10:33:18 2024","interfaces":[{"interface":"back","average":{"1min":1.4330000000000001,"5min":1.4330000000000001,"15min":1.4330000000000001},"min":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"max":{"1min":2.8500000000000001,"5min":2.8500000000000001,"15min":2.8500000000000001},"last":1.5509999999999999},{"interface":"front","average":{"1min":1.4079999999999999,"5min":1.4079999999999999,"15min":1.4079999999999999},"min":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"max":{"1min":2.0070000000000001,"5min":2.0070000000000001,"15min":2.0070000000000001},"last":1.649}]}]},{"osd":8,"up_from":48,"seq":206158430233,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":104857600,"kb_used":298032,"kb_used_data":672,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":104559568,"statfs":{"total":107374182400,"available":107068997632,"internally_reserved":0,"allocated":688128,"data_stored":332771,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[2,3,4,5,6,7,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 10:33:16 2024","interfaces":[{"interface":"back","average":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":1.26,"5min":1.26,"15min":1.26},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":1.238,"5min":1.238,"15min":1.238},"last":0.86299999999999999}]},{"osd":1,"last update":"Wed Sep 18 10:33:16 2024","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.751,"15min":0.751},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":1.129,"5min":1.129,"15min":1.129},"last":1.1359999999999999},{"interface":"front","average":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"min":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.2390000000000001,"5min":1.2390000000000001,"15min":1.2390000000000001},"last":0.92600000000000005}]},{"osd":2,"last update":"Wed Sep 18 10:33:16 2024","interfaces":[{"interface":"back","average":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":1.6439999999999999,"5min":1.6439999999999999,"15min":1.6439999999999999},"last":0.98099999999999998},{"interface":"front","average":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":1.2529999999999999,"5min":1.2529999999999999,"15min":1.2529999999999999},"last":0.70999999999999996}]},{"osd":3,"last update":"Wed Sep 18 10:33:16 2024","interfaces":[{"interface":"back","average":{"1min":1.109,"5min":1.109,"15min":1.109},"min":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"max":{"1min":1.538,"5min":1.538,"15min":1.538},"last":0.78900000000000003},{"interface":"front","average":{"1min":1.0389999999999999,"5min":1.0389999999999999,"15min":1.0389999999999999},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.4810000000000001,"5min":1.4810000000000001,"15min":1.4810000000000001},"last":1.0549999999999999}]},{"osd":4,"last update":"Wed Sep 18 10:33:16 2024","interfaces":[{"interface":"back","average":{"1min":1.0620000000000001,"5min":1.0620000000000001,"15min":1.0620000000000001},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.8169999999999999,"5min":1.8169999999999999,"15min":1.8169999999999999},"last":1.1839999999999999},{"interface":"front","average":{"1min":1.2549999999999999,"5min":1.2549999999999999,"15min":1.2549999999999999},"min":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"max":{"1min":2.1480000000000001,"5min":2.1480000000000001,"15min":2.1480000000000001},"last":1.3220000000000001}]},{"osd":5,"last update":"Wed Sep 18 10:33:16 2024","interfaces":[{"interface":"back","average":{"1min":1.216,"5min":1.216,"15min":1.216},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":2.3109999999999999,"5min":2.3109999999999999,"15min":2.3109999999999999},"last":1.2190000000000001},{"interface":"front","average":{"1min":1.161,"5min":1.161,"15min":1.161},"min":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":2.0169999999999999,"5min":2.0169999999999999,"15min":2.0169999999999999},"last":1.363}]},{"osd":6,"last update":"Wed Sep 18 10:33:16 2024","interfaces":[{"interface":"back","average":{"1min":1.2889999999999999,"5min":1.2889999999999999,"15min":1.2889999999999999},"min":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"max":{"1min":2.1899999999999999,"5min":2.1899999999999999,"15min":2.1899999999999999},"last":1.5329999999999999},{"interface":"front","average":{"1min":1.1799999999999999,"5min":1.1799999999999999,"15min":1.1799999999999999},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":2.069,"5min":2.069,"15min":2.069},"last":1.2529999999999999}]},{"osd":7,"last update":"Wed Sep 18 10:33:16 2024","interfaces":[{"interface":"back","average":{"1min":1.2589999999999999,"5min":1.2589999999999999,"15min":1.2589999999999999},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":2.2130000000000001,"5min":2.2130000000000001,"15min":2.2130000000000001},"last":1.4399999999999999},{"interface":"front","average":{"1min":1.4039999999999999,"5min":1.4039999999999999,"15min":1.4039999999999999},"min":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"max":{"1min":2.2450000000000001,"5min":2.2450000000000001,"15min":2.2450000000000001},"last":1.472}]},{"osd":9,"last update":"Wed Sep 18 10:33:16 2024","interfaces":[{"interface":"back","average":{"1min":1.304,"5min":1.304,"15min":1.304},"min":{"1min":0.63,"5min":0.63,"15min":0.63},"max":{"1min":2.1800000000000002,"5min":2.1800000000000002,"15min":2.1800000000000002},"last":1.403},{"interface":"front","average":{"1min":1.3440000000000001,"5min":1.3440000000000001,"15min":1.3440000000000001},"min":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"max":{"1min":2.3969999999999998,"5min":2.3969999999999998,"15min":2.3969999999999998},"last":1.514}]}]},{"osd":7,"up_from":48,"seq":206158430232,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":298032,"kb_used_data":672,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":104559568,"statfs":{"total":107374182400,"available":107068997632,"internally_reserved":0,"allocated":688128,"data_stored":332771,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[2,3,4,5,6,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 10:33:17 2024","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.202,"5min":1.202,"15min":1.202},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.75,"5min":0.75,"15min":0.75},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":1.486,"5min":1.486,"15min":1.486},"last":0.98799999999999999}]},{"osd":1,"last update":"Wed Sep 18 10:33:17 2024","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.3009999999999999,"5min":1.3009999999999999,"15min":1.3009999999999999},"last":1.085},{"interface":"front","average":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.8480000000000001,"5min":1.8480000000000001,"15min":1.8480000000000001},"last":0.73799999999999999}]},{"osd":2,"last update":"Wed Sep 18 10:33:17 2024","interfaces":[{"interface":"back","average":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":1.5980000000000001,"5min":1.5980000000000001,"15min":1.5980000000000001},"last":1.036},{"interface":"front","average":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.3440000000000001,"5min":1.3440000000000001,"15min":1.3440000000000001},"last":1.212}]},{"osd":3,"last update":"Wed Sep 18 10:33:17 2024","interfaces":[{"interface":"back","average":{"1min":1.1000000000000001,"5min":1.1000000000000001,"15min":1.1000000000000001},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.9319999999999999,"5min":1.9319999999999999,"15min":1.9319999999999999},"last":0.91300000000000003},{"interface":"front","average":{"1min":1.0720000000000001,"5min":1.0720000000000001,"15min":1.0720000000000001},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.8879999999999999,"5min":1.8879999999999999,"15min":1.8879999999999999},"last":1.0600000000000001}]},{"osd":4,"last update":"Wed Sep 18 10:33:17 2024","interfaces":[{"interface":"back","average":{"1min":1.097,"5min":1.097,"15min":1.097},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":2.1379999999999999,"5min":2.1379999999999999,"15min":2.1379999999999999},"last":0.77600000000000002},{"interface":"front","average":{"1min":1.1060000000000001,"5min":1.1060000000000001,"15min":1.1060000000000001},"min":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":2.0110000000000001,"5min":2.0110000000000001,"15min":2.0110000000000001},"last":1.1639999999999999}]},{"osd":5,"last update":"Wed Sep 18 10:33:17 2024","interfaces":[{"interface":"back","average":{"1min":1.194,"5min":1.194,"15min":1.194},"min":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"max":{"1min":2.4889999999999999,"5min":2.4889999999999999,"15min":2.4889999999999999},"last":1.002},{"interface":"front","average":{"1min":1.2230000000000001,"5min":1.2230000000000001,"15min":1.2230000000000001},"min":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"max":{"1min":2.4289999999999998,"5min":2.4289999999999998,"15min":2.4289999999999998},"last":0.86499999999999999}]},{"osd":6,"last update":"Wed Sep 18 10:33:17 2024","interfaces":[{"interface":"back","average":{"1min":1.2769999999999999,"5min":1.2769999999999999,"15min":1.2769999999999999},"min":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"max":{"1min":2.0950000000000002,"5min":2.0950000000000002,"15min":2.0950000000000002},"last":1.284},{"interface":"front","average":{"1min":1.218,"5min":1.218,"15min":1.218},"min":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":1.9450000000000001,"5min":1.9450000000000001,"15min":1.9450000000000001},"last":1.1399999999999999}]},{"osd":8,"last update":"Wed Sep 18 10:33:17 2024","interfaces":[{"interface":"back","average":{"1min":1.2969999999999999,"5min":1.2969999999999999,"15min":1.2969999999999999},"min":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"max":{"1min":2.6389999999999998,"5min":2.6389999999999998,"15min":2.6389999999999998},"last":0.95999999999999996},{"interface":"front","average":{"1min":1.3600000000000001,"5min":1.3600000000000001,"15min":1.3600000000000001},"min":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"max":{"1min":2.1520000000000001,"5min":2.1520000000000001,"15min":2.1520000000000001},"last":1.5229999999999999}]},{"osd":9,"last update":"Wed Sep 18 10:33:17 2024","interfaces":[{"interface":"back","average":{"1min":1.2470000000000001,"5min":1.2470000000000001,"15min":1.2470000000000001},"min":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":2.0680000000000001,"5min":2.0680000000000001,"15min":2.0680000000000001},"last":1.6299999999999999},{"interface":"front","average":{"1min":1.282,"5min":1.282,"15min":1.282},"min":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"max":{"1min":1.8460000000000001,"5min":1.8460000000000001,"15min":1.8460000000000001},"last":1.1040000000000001}]}]},{"osd":6,"up_from":48,"seq":206158430232,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":298032,"kb_used_data":672,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":104559568,"statfs":{"total":107374182400,"available":107068997632,"internally_reserved":0,"allocated":688128,"data_stored":332771,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,7,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 10:33:17 2024","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":1.073,"5min":1.073,"15min":1.073},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":1.3720000000000001,"5min":1.3720000000000001,"15min":1.3720000000000001},"last":0.48499999999999999}]},{"osd":1,"last update":"Wed Sep 18 10:33:17 2024","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.751,"15min":0.751},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":1.6599999999999999,"5min":1.6599999999999999,"15min":1.6599999999999999},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":2.0459999999999998,"5min":2.0459999999999998,"15min":2.0459999999999998},"last":0.72399999999999998}]},{"osd":2,"last update":"Wed Sep 18 10:33:17 2024","interfaces":[{"interface":"back","average":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":1.1399999999999999,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":0.67400000000000004},{"interface":"front","average":{"1min":1.052,"5min":1.052,"15min":1.052},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":1.907,"5min":1.907,"15min":1.907},"last":0.84299999999999997}]},{"osd":3,"last update":"Wed Sep 18 10:33:17 2024","interfaces":[{"interface":"back","average":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.7490000000000001,"5min":1.7490000000000001,"15min":1.7490000000000001},"last":0.78600000000000003},{"interface":"front","average":{"1min":1.1830000000000001,"5min":1.1830000000000001,"15min":1.1830000000000001},"min":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"max":{"1min":1.8680000000000001,"5min":1.8680000000000001,"15min":1.8680000000000001},"last":1.0760000000000001}]},{"osd":4,"last update":"Wed Sep 18 10:33:17 2024","interfaces":[{"interface":"back","average":{"1min":1.2070000000000001,"5min":1.2070000000000001,"15min":1.2070000000000001},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.976,"5min":1.976,"15min":1.976},"last":1.153},{"interface":"front","average":{"1min":1.127,"5min":1.127,"15min":1.127},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":2.544,"5min":2.544,"15min":2.544},"last":1.427}]},{"osd":5,"last update":"Wed Sep 18 10:33:17 2024","interfaces":[{"interface":"back","average":{"1min":1.2170000000000001,"5min":1.2170000000000001,"15min":1.2170000000000001},"min":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"max":{"1min":2.1539999999999999,"5min":2.1539999999999999,"15min":2.1539999999999999},"last":1.1899999999999999},{"interface":"front","average":{"1min":1.296,"5min":1.296,"15min":1.296},"min":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":2.2469999999999999,"5min":2.2469999999999999,"15min":2.2469999999999999},"last":1.5509999999999999}]},{"osd":7,"last update":"Wed Sep 18 10:33:17 2024","interfaces":[{"interface":"back","average":{"1min":1.246,"5min":1.246,"15min":1.246},"min":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":2.6440000000000001,"5min":2.6440000000000001,"15min":2.6440000000000001},"last":1.044},{"interface":"front","average":{"1min":1.7609999999999999,"5min":1.7609999999999999,"15min":1.7609999999999999},"min":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"max":{"1min":10.385,"5min":10.385,"15min":10.385},"last":1.3240000000000001}]},{"osd":8,"last update":"Wed Sep 18 10:33:17 2024","interfaces":[{"interface":"back","average":{"1min":1.3640000000000001,"5min":1.3640000000000001,"15min":1.3640000000000001},"min":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"max":{"1min":2.444,"5min":2.444,"15min":2.444},"last":1.6180000000000001},{"interface":"front","average":{"1min":1.274,"5min":1.274,"15min":1.274},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":2.714,"5min":2.714,"15min":2.714},"last":1.1100000000000001}]},{"osd":9,"last update":"Wed Sep 18 10:33:17 2024","interfaces":[{"interface":"back","average":{"1min":1.2789999999999999,"5min":1.2789999999999999,"15min":1.2789999999999999},"min":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"max":{"1min":2.2879999999999998,"5min":2.2879999999999998,"15min":2.2879999999999998},"last":1.3620000000000001},{"interface":"front","average":{"1min":1.3240000000000001,"5min":1.3240000000000001,"15min":1.3240000000000001},"min":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":2.5960000000000001,"5min":2.5960000000000001,"15min":2.5960000000000001},"last":1.5860000000000001}]}]},{"osd":5,"up_from":48,"seq":206158430234,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":298028,"kb_used_data":672,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":104559572,"statfs":{"total":107374182400,"available":107069001728,"internally_reserved":0,"allocated":688128,"data_stored":332771,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,6,7,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 10:33:18 2024","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.111,"5min":1.111,"15min":1.111},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"min":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.5249999999999999,"5min":1.5249999999999999,"15min":1.5249999999999999},"last":0.67700000000000005}]},{"osd":1,"last update":"Wed Sep 18 10:33:18 2024","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.173,"5min":1.173,"15min":1.173},"last":1.02},{"interface":"front","average":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.8280000000000001,"5min":1.8280000000000001,"15min":1.8280000000000001},"last":1.1599999999999999}]},{"osd":2,"last update":"Wed Sep 18 10:33:18 2024","interfaces":[{"interface":"back","average":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"min":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":1.4159999999999999,"5min":1.4159999999999999,"15min":1.4159999999999999},"last":1.194},{"interface":"front","average":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"min":{"1min":0.627,"5min":0.627,"15min":0.627},"max":{"1min":2.3849999999999998,"5min":2.3849999999999998,"15min":2.3849999999999998},"last":1.0880000000000001}]},{"osd":3,"last update":"Wed Sep 18 10:33:18 2024","interfaces":[{"interface":"back","average":{"1min":1.079,"5min":1.079,"15min":1.079},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.923,"5min":1.923,"15min":1.923},"last":1.4139999999999999},{"interface":"front","average":{"1min":1.268,"5min":1.268,"15min":1.268},"min":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"max":{"1min":2.073,"5min":2.073,"15min":2.073},"last":1.5269999999999999}]},{"osd":4,"last update":"Wed Sep 18 10:33:18 2024","interfaces":[{"interface":"back","average":{"1min":1.123,"5min":1.123,"15min":1.123},"min":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"max":{"1min":2.141,"5min":2.141,"15min":2.141},"last":1.25},{"interface":"front","average":{"1min":1.2490000000000001,"5min":1.2490000000000001,"15min":1.2490000000000001},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":2.3300000000000001,"5min":2.3300000000000001,"15min":2.3300000000000001},"last":1.569}]},{"osd":6,"last update":"Wed Sep 18 10:33:18 2024","interfaces":[{"interface":"back","average":{"1min":1.2769999999999999,"5min":1.2769999999999999,"15min":1.2769999999999999},"min":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"max":{"1min":1.806,"5min":1.806,"15min":1.806},"last":1.2909999999999999},{"interface":"front","average":{"1min":1.274,"5min":1.274,"15min":1.274},"min":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"max":{"1min":2.6120000000000001,"5min":2.6120000000000001,"15min":2.6120000000000001},"last":1.05}]},{"osd":7,"last update":"Wed Sep 18 10:33:18 2024","interfaces":[{"interface":"back","average":{"1min":1.3380000000000001,"5min":1.3380000000000001,"15min":1.3380000000000001},"min":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"max":{"1min":2.661,"5min":2.661,"15min":2.661},"last":1.49},{"interface":"front","average":{"1min":1.3819999999999999,"5min":1.3819999999999999,"15min":1.3819999999999999},"min":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"max":{"1min":2.548,"5min":2.548,"15min":2.548},"last":1.3460000000000001}]},{"osd":8,"last update":"Wed Sep 18 10:33:18 2024","interfaces":[{"interface":"back","average":{"1min":1.359,"5min":1.359,"15min":1.359},"min":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"max":{"1min":2.7080000000000002,"5min":2.7080000000000002,"15min":2.7080000000000002},"last":1.4630000000000001},{"interface":"front","average":{"1min":1.327,"5min":1.327,"15min":1.327},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":2.4399999999999999,"5min":2.4399999999999999,"15min":2.4399999999999999},"last":1.645}]},{"osd":9,"last update":"Wed Sep 18 10:33:18 2024","interfaces":[{"interface":"back","average":{"1min":1.3859999999999999,"5min":1.3859999999999999,"15min":1.3859999999999999},"min":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"max":{"1min":2.2130000000000001,"5min":2.2130000000000001,"15min":2.2130000000000001},"last":1.1259999999999999},{"interface":"front","average":{"1min":1.4019999999999999,"5min":1.4019999999999999,"15min":1.4019999999999999},"min":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"max":{"1min":2.782,"5min":2.782,"15min":2.782},"last":1.3680000000000001}]}]},{"osd":4,"up_from":48,"seq":206158430232,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":298036,"kb_used_data":676,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":104559564,"statfs":{"total":107374182400,"available":107068993536,"internally_reserved":0,"allocated":692224,"data_stored":332790,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,5,6,7,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 10:33:16 2024","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":1.0960000000000001,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.745,"5min":0.745,"15min":0.745},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.171,"5min":1.171,"15min":1.171},"last":0.57499999999999996}]},{"osd":1,"last update":"Wed Sep 18 10:33:16 2024","interfaces":[{"interface":"back","average":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":1.1579999999999999,"5min":1.1579999999999999,"15min":1.1579999999999999},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.583,"5min":1.583,"15min":1.583},"last":1.038}]},{"osd":2,"last update":"Wed Sep 18 10:33:16 2024","interfaces":[{"interface":"back","average":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":2.1139999999999999,"5min":2.1139999999999999,"15min":2.1139999999999999},"last":1.0840000000000001},{"interface":"front","average":{"1min":0.995,"5min":0.995,"15min":0.995},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.5660000000000001,"5min":1.5660000000000001,"15min":1.5660000000000001},"last":1.302}]},{"osd":3,"last update":"Wed Sep 18 10:33:16 2024","interfaces":[{"interface":"back","average":{"1min":1.0620000000000001,"5min":1.0620000000000001,"15min":1.0620000000000001},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.6200000000000001,"5min":1.6200000000000001,"15min":1.6200000000000001},"last":0.84299999999999997},{"interface":"front","average":{"1min":1.1699999999999999,"5min":1.1699999999999999,"15min":1.1699999999999999},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.732,"5min":1.732,"15min":1.732},"last":1.337}]},{"osd":5,"last update":"Wed Sep 18 10:33:16 2024","interfaces":[{"interface":"back","average":{"1min":1.159,"5min":1.159,"15min":1.159},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":2.2509999999999999,"5min":2.2509999999999999,"15min":2.2509999999999999},"last":1.421},{"interface":"front","average":{"1min":1.1830000000000001,"5min":1.1830000000000001,"15min":1.1830000000000001},"min":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":1.873,"5min":1.873,"15min":1.873},"last":1.2070000000000001}]},{"osd":6,"last update":"Wed Sep 18 10:33:16 2024","interfaces":[{"interface":"back","average":{"1min":1.214,"5min":1.214,"15min":1.214},"min":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"max":{"1min":2.2879999999999998,"5min":2.2879999999999998,"15min":2.2879999999999998},"last":0.89500000000000002},{"interface":"front","average":{"1min":1.25,"5min":1.25,"15min":1.25},"min":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":2.661,"5min":2.661,"15min":2.661},"last":2.0750000000000002}]},{"osd":7,"last update":"Wed Sep 18 10:33:16 2024","interfaces":[{"interface":"back","average":{"1min":1.319,"5min":1.319,"15min":1.319},"min":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"max":{"1min":1.895,"5min":1.895,"15min":1.895},"last":2.7919999999999998},{"interface":"front","average":{"1min":1.2450000000000001,"5min":1.2450000000000001,"15min":1.2450000000000001},"min":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":2.0579999999999998,"5min":2.0579999999999998,"15min":2.0579999999999998},"last":1.5649999999999999}]},{"osd":8,"last update":"Wed Sep 18 10:33:16 2024","interfaces":[{"interface":"back","average":{"1min":1.278,"5min":1.278,"15min":1.278},"min":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":2.173,"5min":2.173,"15min":2.173},"last":1.3720000000000001},{"interface":"front","average":{"1min":1.3240000000000001,"5min":1.3240000000000001,"15min":1.3240000000000001},"min":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":1.9750000000000001,"5min":1.9750000000000001,"15min":1.9750000000000001},"last":1.7949999999999999}]},{"osd":9,"last update":"Wed Sep 18 10:33:16 2024","interfaces":[{"interface":"back","average":{"1min":1.327,"5min":1.327,"15min":1.327},"min":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":2.448,"5min":2.448,"15min":2.448},"last":1.704},{"interface":"front","average":{"1min":1.389,"5min":1.389,"15min":1.389},"min":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"max":{"1min":2.242,"5min":2.242,"15min":2.242},"last":2.1619999999999999}]}]},{"osd":3,"up_from":48,"seq":206158430234,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":298032,"kb_used_data":672,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":104559568,"statfs":{"total":107374182400,"available":107068997632,"internally_reserved":0,"allocated":688128,"data_stored":332771,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,4,5,6,7,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 10:33:15 2024","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.095,"5min":1.095,"15min":1.095},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"min":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":2.121,"5min":2.121,"15min":2.121},"last":1.2450000000000001}]},{"osd":1,"last update":"Wed Sep 18 10:33:15 2024","interfaces":[{"interface":"back","average":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":1.9259999999999999,"5min":1.9259999999999999,"15min":1.9259999999999999},"last":0.83099999999999996},{"interface":"front","average":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"min":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":1.2090000000000001,"5min":1.2090000000000001,"15min":1.2090000000000001},"last":1.3819999999999999}]},{"osd":2,"last update":"Wed Sep 18 10:33:15 2024","interfaces":[{"interface":"back","average":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.204,"5min":1.204,"15min":1.204},"last":0.624},{"interface":"front","average":{"1min":1.1870000000000001,"5min":1.1870000000000001,"15min":1.1870000000000001},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":3.2040000000000002,"5min":3.2040000000000002,"15min":3.2040000000000002},"last":0.59699999999999998}]},{"osd":4,"last update":"Wed Sep 18 10:33:15 2024","interfaces":[{"interface":"back","average":{"1min":1.071,"5min":1.071,"15min":1.071},"min":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":1.49,"5min":1.49,"15min":1.49},"last":0.75900000000000001},{"interface":"front","average":{"1min":1.266,"5min":1.266,"15min":1.266},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":2.7519999999999998,"5min":2.7519999999999998,"15min":2.7519999999999998},"last":1.3200000000000001}]},{"osd":5,"last update":"Wed Sep 18 10:33:15 2024","interfaces":[{"interface":"back","average":{"1min":1.1830000000000001,"5min":1.1830000000000001,"15min":1.1830000000000001},"min":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"max":{"1min":2.0630000000000002,"5min":2.0630000000000002,"15min":2.0630000000000002},"last":1.145},{"interface":"front","average":{"1min":1.1259999999999999,"5min":1.1259999999999999,"15min":1.1259999999999999},"min":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"max":{"1min":1.5149999999999999,"5min":1.5149999999999999,"15min":1.5149999999999999},"last":0.71699999999999997}]},{"osd":6,"last update":"Wed Sep 18 10:33:15 2024","interfaces":[{"interface":"back","average":{"1min":1.3620000000000001,"5min":1.3620000000000001,"15min":1.3620000000000001},"min":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":4.1029999999999998,"5min":4.1029999999999998,"15min":4.1029999999999998},"last":1.0389999999999999},{"interface":"front","average":{"1min":1.268,"5min":1.268,"15min":1.268},"min":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"max":{"1min":2.2789999999999999,"5min":2.2789999999999999,"15min":2.2789999999999999},"last":0.95399999999999996}]},{"osd":7,"last update":"Wed Sep 18 10:33:15 2024","interfaces":[{"interface":"back","average":{"1min":1.2010000000000001,"5min":1.2010000000000001,"15min":1.2010000000000001},"min":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"max":{"1min":2.1880000000000002,"5min":2.1880000000000002,"15min":2.1880000000000002},"last":0.89000000000000001},{"interface":"front","average":{"1min":1.282,"5min":1.282,"15min":1.282},"min":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"max":{"1min":3.0409999999999999,"5min":3.0409999999999999,"15min":3.0409999999999999},"last":1.429}]},{"osd":8,"last update":"Wed Sep 18 10:33:15 2024","interfaces":[{"interface":"back","average":{"1min":1.5109999999999999,"5min":1.5109999999999999,"15min":1.5109999999999999},"min":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"max":{"1min":3.5590000000000002,"5min":3.5590000000000002,"15min":3.5590000000000002},"last":1.476},{"interface":"front","average":{"1min":1.3540000000000001,"5min":1.3540000000000001,"15min":1.3540000000000001},"min":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"max":{"1min":2.3919999999999999,"5min":2.3919999999999999,"15min":2.3919999999999999},"last":1.3600000000000001}]},{"osd":9,"last update":"Wed Sep 18 10:33:15 2024","interfaces":[{"interface":"back","average":{"1min":1.379,"5min":1.379,"15min":1.379},"min":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"max":{"1min":3.1259999999999999,"5min":3.1259999999999999,"15min":3.1259999999999999},"last":1.5169999999999999},{"interface":"front","average":{"1min":1.5109999999999999,"5min":1.5109999999999999,"15min":1.5109999999999999},"min":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"max":{"1min":3.6339999999999999,"5min":3.6339999999999999,"15min":3.6339999999999999},"last":1.292}]}]},{"osd":2,"up_from":48,"seq":206158430232,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":104857600,"kb_used":298028,"kb_used_data":672,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":104559572,"statfs":{"total":107374182400,"available":107069001728,"internally_reserved":0,"allocated":688128,"data_stored":332771,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,3,4,5,6,7,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 10:33:18 2024","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1.109,"5min":1.109,"15min":1.109},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":1.5029999999999999,"5min":1.5029999999999999,"15min":1.5029999999999999},"last":1.119}]},{"osd":1,"last update":"Wed Sep 18 10:33:18 2024","interfaces":[{"interface":"back","average":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"min":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.651,"5min":1.651,"15min":1.651},"last":0.82599999999999996},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":1.1559999999999999,"5min":1.1559999999999999,"15min":1.1559999999999999},"last":1.2769999999999999}]},{"osd":3,"last update":"Wed Sep 18 10:33:18 2024","interfaces":[{"interface":"back","average":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":1.2869999999999999,"5min":1.2869999999999999,"15min":1.2869999999999999},"last":0.92200000000000004},{"interface":"front","average":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.7390000000000001,"5min":1.7390000000000001,"15min":1.7390000000000001},"last":0.64100000000000001}]},{"osd":4,"last update":"Wed Sep 18 10:33:18 2024","interfaces":[{"interface":"back","average":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"min":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":1.333,"5min":1.333,"15min":1.333},"last":1.3540000000000001},{"interface":"front","average":{"1min":1.151,"5min":1.151,"15min":1.151},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.7869999999999999,"5min":1.7869999999999999,"15min":1.7869999999999999},"last":1.032}]},{"osd":5,"last update":"Wed Sep 18 10:33:18 2024","interfaces":[{"interface":"back","average":{"1min":1.1100000000000001,"5min":1.1100000000000001,"15min":1.1100000000000001},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.597,"5min":1.597,"15min":1.597},"last":1.417},{"interface":"front","average":{"1min":1.246,"5min":1.246,"15min":1.246},"min":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"max":{"1min":1.873,"5min":1.873,"15min":1.873},"last":1.159}]},{"osd":6,"last update":"Wed Sep 18 10:33:18 2024","interfaces":[{"interface":"back","average":{"1min":1.0649999999999999,"5min":1.0649999999999999,"15min":1.0649999999999999},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":1.696,"5min":1.696,"15min":1.696},"last":1.093},{"interface":"front","average":{"1min":1.1479999999999999,"5min":1.1479999999999999,"15min":1.1479999999999999},"min":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"max":{"1min":1.5289999999999999,"5min":1.5289999999999999,"15min":1.5289999999999999},"last":1.8899999999999999}]},{"osd":7,"last update":"Wed Sep 18 10:33:18 2024","interfaces":[{"interface":"back","average":{"1min":1.268,"5min":1.268,"15min":1.268},"min":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":1.9350000000000001,"5min":1.9350000000000001,"15min":1.9350000000000001},"last":0.97299999999999998},{"interface":"front","average":{"1min":1.244,"5min":1.244,"15min":1.244},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":2.6419999999999999,"5min":2.6419999999999999,"15min":2.6419999999999999},"last":1.8069999999999999}]},{"osd":8,"last update":"Wed Sep 18 10:33:18 2024","interfaces":[{"interface":"back","average":{"1min":1.306,"5min":1.306,"15min":1.306},"min":{"1min":1,"5min":1,"15min":1},"max":{"1min":1.613,"5min":1.613,"15min":1.613},"last":1.3160000000000001},{"interface":"front","average":{"1min":1.278,"5min":1.278,"15min":1.278},"min":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":1.9830000000000001,"5min":1.9830000000000001,"15min":1.9830000000000001},"last":1.3779999999999999}]},{"osd":9,"last update":"Wed Sep 18 10:33:18 2024","interfaces":[{"interface":"back","average":{"1min":1.2210000000000001,"5min":1.2210000000000001,"15min":1.2210000000000001},"min":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"max":{"1min":2.4220000000000002,"5min":2.4220000000000002,"15min":2.4220000000000002},"last":2.0699999999999998},{"interface":"front","average":{"1min":1.337,"5min":1.337,"15min":1.337},"min":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"max":{"1min":2.1349999999999998,"5min":2.1349999999999998,"15min":2.1349999999999998},"last":1.24}]}]},{"osd":1,"up_from":59,"seq":253403070489,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":298040,"kb_used_data":676,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":104559560,"statfs":{"total":107374182400,"available":107068989440,"internally_reserved":0,"allocated":692224,"data_stored":332790,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,2,3,4,5,6,7,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 10:33:17 2024","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":1.0980000000000001,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":1.3979999999999999},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"min":{"1min":0.248,"5min":0.248,"15min":0.248},"max":{"1min":1.3,"5min":1.3,"15min":1.3},"last":0.85299999999999998}]},{"osd":2,"last update":"Wed Sep 18 10:33:17 2024","interfaces":[{"interface":"back","average":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.206,"5min":1.206,"15min":1.206},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":1.54,"5min":1.54,"15min":1.54},"last":2.23}]},{"osd":3,"last update":"Wed Sep 18 10:33:17 2024","interfaces":[{"interface":"back","average":{"1min":1.129,"5min":1.129,"15min":1.129},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":5.7569999999999997,"5min":5.7569999999999997,"15min":5.7569999999999997},"last":0.94299999999999995},{"interface":"front","average":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":2.9830000000000001,"5min":2.9830000000000001,"15min":2.9830000000000001},"last":0.70199999999999996}]},{"osd":4,"last update":"Wed Sep 18 10:33:17 2024","interfaces":[{"interface":"back","average":{"1min":1.1000000000000001,"5min":1.1000000000000001,"15min":1.1000000000000001},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":3.8700000000000001,"5min":3.8700000000000001,"15min":3.8700000000000001},"last":0.73399999999999999},{"interface":"front","average":{"1min":1.1359999999999999,"5min":1.1359999999999999,"15min":1.1359999999999999},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":2.6520000000000001,"5min":2.6520000000000001,"15min":2.6520000000000001},"last":2.0670000000000002}]},{"osd":5,"last update":"Wed Sep 18 10:33:17 2024","interfaces":[{"interface":"back","average":{"1min":1.1160000000000001,"5min":1.1160000000000001,"15min":1.1160000000000001},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":2.0470000000000002,"5min":2.0470000000000002,"15min":2.0470000000000002},"last":2.1560000000000001},{"interface":"front","average":{"1min":1.0920000000000001,"5min":1.0920000000000001,"15min":1.0920000000000001},"min":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"max":{"1min":1.6679999999999999,"5min":1.6679999999999999,"15min":1.6679999999999999},"last":1.0409999999999999}]},{"osd":6,"last update":"Wed Sep 18 10:33:17 2024","interfaces":[{"interface":"back","average":{"1min":1.3280000000000001,"5min":1.3280000000000001,"15min":1.3280000000000001},"min":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":3.1299999999999999,"5min":3.1299999999999999,"15min":3.1299999999999999},"last":1.125},{"interface":"front","average":{"1min":1.123,"5min":1.123,"15min":1.123},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.9390000000000001,"5min":1.9390000000000001,"15min":1.9390000000000001},"last":0.97799999999999998}]},{"osd":7,"last update":"Wed Sep 18 10:33:17 2024","interfaces":[{"interface":"back","average":{"1min":1.1060000000000001,"5min":1.1060000000000001,"15min":1.1060000000000001},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":2.4900000000000002,"5min":2.4900000000000002,"15min":2.4900000000000002},"last":1.256},{"interface":"front","average":{"1min":1.3120000000000001,"5min":1.3120000000000001,"15min":1.3120000000000001},"min":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":3.2810000000000001,"5min":3.2810000000000001,"15min":3.2810000000000001},"last":2.3010000000000002}]},{"osd":8,"last update":"Wed Sep 18 10:33:17 2024","interfaces":[{"interface":"back","average":{"1min":1.258,"5min":1.258,"15min":1.258},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":4.4969999999999999,"5min":4.4969999999999999,"15min":4.4969999999999999},"last":1.3089999999999999},{"interface":"front","average":{"1min":1.1419999999999999,"5min":1.1419999999999999,"15min":1.1419999999999999},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":2.1629999999999998,"5min":2.1629999999999998,"15min":2.1629999999999998},"last":1.169}]},{"osd":9,"last update":"Wed Sep 18 10:33:17 2024","interfaces":[{"interface":"back","average":{"1min":1.4419999999999999,"5min":1.4419999999999999,"15min":1.4419999999999999},"min":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"max":{"1min":3.2160000000000002,"5min":3.2160000000000002,"15min":3.2160000000000002},"last":2.3799999999999999},{"interface":"front","average":{"1min":1.2410000000000001,"5min":1.2410000000000001,"15min":1.2410000000000001},"min":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"max":{"1min":1.9430000000000001,"5min":1.9430000000000001,"15min":1.9430000000000001},"last":1.202}]}]},{"osd":0,"up_from":59,"seq":253403070489,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":298036,"kb_used_data":672,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":104559564,"statfs":{"total":107374182400,"available":107068993536,"internally_reserved":0,"allocated":688128,"data_stored":332771,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[1,2,3,4,5,6,7,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Sep 18 10:33:15 2024","interfaces":[{"interface":"back","average":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":1.5880000000000001,"5min":1.5880000000000001,"15min":1.5880000000000001},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.917,"5min":1.917,"15min":1.917},"last":2.218}]},{"osd":2,"last update":"Wed Sep 18 10:33:15 2024","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.746,"15min":0.746},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.454,"5min":1.454,"15min":1.454},"last":1.5409999999999999},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"min":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.222,"5min":1.222,"15min":1.222},"last":0.57399999999999995}]},{"osd":3,"last update":"Wed Sep 18 10:33:15 2024","interfaces":[{"interface":"back","average":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":1.5509999999999999,"5min":1.5509999999999999,"15min":1.5509999999999999},"last":1.0589999999999999},{"interface":"front","average":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.6339999999999999,"5min":1.6339999999999999,"15min":1.6339999999999999},"last":2.137}]},{"osd":4,"last update":"Wed Sep 18 10:33:15 2024","interfaces":[{"interface":"back","average":{"1min":1,"5min":1,"15min":1},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":1.6339999999999999,"5min":1.6339999999999999,"15min":1.6339999999999999},"last":0.91000000000000003},{"interface":"front","average":{"1min":1.093,"5min":1.093,"15min":1.093},"min":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.704,"5min":1.704,"15min":1.704},"last":1.1399999999999999}]},{"osd":5,"last update":"Wed Sep 18 10:33:15 2024","interfaces":[{"interface":"back","average":{"1min":1.1699999999999999,"5min":1.1699999999999999,"15min":1.1699999999999999},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.9590000000000001,"5min":1.9590000000000001,"15min":1.9590000000000001},"last":2.2829999999999999},{"interface":"front","average":{"1min":1.2050000000000001,"5min":1.2050000000000001,"15min":1.2050000000000001},"min":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"max":{"1min":2.206,"5min":2.206,"15min":2.206},"last":1.4330000000000001}]},{"osd":6,"last update":"Wed Sep 18 10:33:15 2024","interfaces":[{"interface":"back","average":{"1min":1.157,"5min":1.157,"15min":1.157},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":1.74,"5min":1.74,"15min":1.74},"last":1.974},{"interface":"front","average":{"1min":1.135,"5min":1.135,"15min":1.135},"min":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"max":{"1min":2.8140000000000001,"5min":2.8140000000000001,"15min":2.8140000000000001},"last":0.998}]},{"osd":7,"last update":"Wed Sep 18 10:33:15 2024","interfaces":[{"interface":"back","average":{"1min":1.268,"5min":1.268,"15min":1.268},"min":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"max":{"1min":2.004,"5min":2.004,"15min":2.004},"last":1.673},{"interface":"front","average":{"1min":1.357,"5min":1.357,"15min":1.357},"min":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"max":{"1min":2.7250000000000001,"5min":2.7250000000000001,"15min":2.7250000000000001},"last":2.0630000000000002}]},{"osd":8,"last update":"Wed Sep 18 10:33:15 2024","interfaces":[{"interface":"back","average":{"1min":1.3049999999999999,"5min":1.3049999999999999,"15min":1.3049999999999999},"min":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":2.1419999999999999,"5min":2.1419999999999999,"15min":2.1419999999999999},"last":1.837},{"interface":"front","average":{"1min":1.3779999999999999,"5min":1.3779999999999999,"15min":1.3779999999999999},"min":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"max":{"1min":2.8980000000000001,"5min":2.8980000000000001,"15min":2.8980000000000001},"last":2.6379999999999999}]},{"osd":9,"last update":"Wed Sep 18 10:33:15 2024","interfaces":[{"interface":"back","average":{"1min":1.2989999999999999,"5min":1.2989999999999999,"15min":1.2989999999999999},"min":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"max":{"1min":2.3109999999999999,"5min":2.3109999999999999,"15min":2.3109999999999999},"last":1.1080000000000001},{"interface":"front","average":{"1min":1.333,"5min":1.333,"15min":1.333},"min":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":2.2639999999999998,"5min":2.2639999999999998,"15min":2.2639999999999998},"last":1.198}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-18T10:34:08.233 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-18T10:34:08.233 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-18T10:34:08.522 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-18T10:34:08.525 INFO:teuthology.orchestra.run.smithi134.stderr:dumped all 2024-09-18T10:34:08.537 INFO:teuthology.orchestra.run.smithi134.stdout:{"pg_ready":true,"pg_map":{"version":107,"stamp":"2024-09-18T10:34:07.432291+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":10,"num_per_pool_osds":10,"num_per_pool_omap_osds":8,"kb":1048576000,"kb_used":2980332,"kb_used_data":6728,"kb_used_omap":16,"kb_used_meta":2973423,"kb_avail":1045595668,"statfs":{"total":1073741824000,"available":1070689964032,"internally_reserved":0,"allocated":6889472,"data_stored":3327748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":17040,"internal_metadata":3044785520},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-1,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.324608"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":47,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-18T10:34:02.428277+0000","last_change":"2024-09-18T10:33:52.036028+0000","last_active":"2024-09-18T10:34:02.428277+0000","last_peered":"2024-09-18T10:34:02.428277+0000","last_clean":"2024-09-18T10:34:02.428277+0000","last_became_active":"2024-09-18T10:33:52.035591+0000","last_became_peered":"2024-09-18T10:33:52.035591+0000","last_unstale":"2024-09-18T10:34:02.428277+0000","last_undegraded":"2024-09-18T10:34:02.428277+0000","last_fullsized":"2024-09-18T10:34:02.428277+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T10:32:19.443095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:32:19.443095+0000","last_clean_scrub_stamp":"2024-09-18T10:32:19.443095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T18:06:23.838644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00038275899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":63,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-18T10:34:02.778343+0000","last_change":"2024-09-18T10:33:52.042125+0000","last_active":"2024-09-18T10:34:02.778343+0000","last_peered":"2024-09-18T10:34:02.778343+0000","last_clean":"2024-09-18T10:34:02.778343+0000","last_became_active":"2024-09-18T10:33:52.041895+0000","last_became_peered":"2024-09-18T10:33:52.041895+0000","last_unstale":"2024-09-18T10:34:02.778343+0000","last_undegraded":"2024-09-18T10:34:02.778343+0000","last_fullsized":"2024-09-18T10:34:02.778343+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T10:32:19.443095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:32:19.443095+0000","last_clean_scrub_stamp":"2024-09-18T10:32:19.443095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T15:18:02.048451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034975400000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":96,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-18T10:34:03.594986+0000","last_change":"2024-09-18T10:34:03.594986+0000","last_active":"2024-09-18T10:34:03.594986+0000","last_peered":"2024-09-18T10:34:03.594986+0000","last_clean":"2024-09-18T10:34:03.594986+0000","last_became_active":"2024-09-18T10:34:03.594428+0000","last_became_peered":"2024-09-18T10:34:03.594428+0000","last_unstale":"2024-09-18T10:34:03.594986+0000","last_undegraded":"2024-09-18T10:34:03.594986+0000","last_fullsized":"2024-09-18T10:34:03.594986+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T10:32:19.443095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:32:19.443095+0000","last_clean_scrub_stamp":"2024-09-18T10:32:19.443095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T18:25:37.752825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048480000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0],"acting":[9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":61,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-18T10:34:03.531946+0000","last_change":"2024-09-18T10:34:03.531946+0000","last_active":"2024-09-18T10:34:03.531946+0000","last_peered":"2024-09-18T10:34:03.531946+0000","last_clean":"2024-09-18T10:34:03.531946+0000","last_became_active":"2024-09-18T10:34:03.531651+0000","last_became_peered":"2024-09-18T10:34:03.531651+0000","last_unstale":"2024-09-18T10:34:03.531946+0000","last_undegraded":"2024-09-18T10:34:03.531946+0000","last_fullsized":"2024-09-18T10:34:03.531946+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T10:32:19.443095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:32:19.443095+0000","last_clean_scrub_stamp":"2024-09-18T10:32:19.443095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T19:47:09.963588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.3","version":"18'1","reported_seq":82,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-18T10:34:03.137967+0000","last_change":"2024-09-18T10:34:03.137967+0000","last_active":"2024-09-18T10:34:03.137967+0000","last_peered":"2024-09-18T10:34:03.137967+0000","last_clean":"2024-09-18T10:34:03.137967+0000","last_became_active":"2024-09-18T10:34:03.137466+0000","last_became_peered":"2024-09-18T10:34:03.137466+0000","last_unstale":"2024-09-18T10:34:03.137967+0000","last_undegraded":"2024-09-18T10:34:03.137967+0000","last_fullsized":"2024-09-18T10:34:03.137967+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T10:32:19.443095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:32:19.443095+0000","last_clean_scrub_stamp":"2024-09-18T10:32:19.443095+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T19:22:54.145211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.2","version":"20'2","reported_seq":85,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-18T10:34:03.136912+0000","last_change":"2024-09-18T10:34:03.136912+0000","last_active":"2024-09-18T10:34:03.136912+0000","last_peered":"2024-09-18T10:34:03.136912+0000","last_clean":"2024-09-18T10:34:03.136912+0000","last_became_active":"2024-09-18T10:34:03.136418+0000","last_became_peered":"2024-09-18T10:34:03.136418+0000","last_unstale":"2024-09-18T10:34:03.136912+0000","last_undegraded":"2024-09-18T10:34:03.136912+0000","last_fullsized":"2024-09-18T10:34:03.136912+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T10:32:19.443095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:32:19.443095+0000","last_clean_scrub_stamp":"2024-09-18T10:32:19.443095+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T21:39:17.714747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019475040000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":44,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-18T10:34:02.562266+0000","last_change":"2024-09-18T10:33:52.038532+0000","last_active":"2024-09-18T10:34:02.562266+0000","last_peered":"2024-09-18T10:34:02.562266+0000","last_clean":"2024-09-18T10:34:02.562266+0000","last_became_active":"2024-09-18T10:33:52.038239+0000","last_became_peered":"2024-09-18T10:33:52.038239+0000","last_unstale":"2024-09-18T10:34:02.562266+0000","last_undegraded":"2024-09-18T10:34:02.562266+0000","last_fullsized":"2024-09-18T10:34:02.562266+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T10:32:19.443095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:32:19.443095+0000","last_clean_scrub_stamp":"2024-09-18T10:32:19.443095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T18:34:12.717302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062722600000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":63,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-18T10:34:00.270430+0000","last_change":"2024-09-18T10:33:52.037375+0000","last_active":"2024-09-18T10:34:00.270430+0000","last_peered":"2024-09-18T10:34:00.270430+0000","last_clean":"2024-09-18T10:34:00.270430+0000","last_became_active":"2024-09-18T10:33:52.037028+0000","last_became_peered":"2024-09-18T10:33:52.037028+0000","last_unstale":"2024-09-18T10:34:00.270430+0000","last_undegraded":"2024-09-18T10:34:00.270430+0000","last_fullsized":"2024-09-18T10:34:00.270430+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T10:32:19.443095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:32:19.443095+0000","last_clean_scrub_stamp":"2024-09-18T10:32:19.443095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T19:41:50.364436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9],"acting":[5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":9}],"osd_stats":[{"osd":9,"up_from":41,"seq":176093659160,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":298036,"kb_used_data":672,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":104559564,"statfs":{"total":107374182400,"available":107068993536,"internally_reserved":0,"allocated":688128,"data_stored":332771,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,6,7,8],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 10:33:18 2024","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":1.1930000000000001,"5min":1.1930000000000001,"15min":1.1930000000000001},"last":0.91700000000000004},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.3029999999999999,"5min":1.3029999999999999,"15min":1.3029999999999999},"last":0.79600000000000004}]},{"osd":1,"last update":"Wed Sep 18 10:33:18 2024","interfaces":[{"interface":"back","average":{"1min":0.873,"5min":0.873,"15min":0.873},"min":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.3919999999999999,"5min":1.3919999999999999,"15min":1.3919999999999999},"last":0.83599999999999997},{"interface":"front","average":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":2.4359999999999999,"5min":2.4359999999999999,"15min":2.4359999999999999},"last":1.149}]},{"osd":2,"last update":"Wed Sep 18 10:33:18 2024","interfaces":[{"interface":"back","average":{"1min":1.03,"5min":1.03,"15min":1.03},"min":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":1.526,"5min":1.526,"15min":1.526},"last":0.69899999999999995},{"interface":"front","average":{"1min":1.044,"5min":1.044,"15min":1.044},"min":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"max":{"1min":1.591,"5min":1.591,"15min":1.591},"last":1.4590000000000001}]},{"osd":3,"last update":"Wed Sep 18 10:33:18 2024","interfaces":[{"interface":"back","average":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.542,"5min":1.542,"15min":1.542},"last":1.1830000000000001},{"interface":"front","average":{"1min":1.274,"5min":1.274,"15min":1.274},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.8440000000000001,"5min":1.8440000000000001,"15min":1.8440000000000001},"last":1.3979999999999999}]},{"osd":4,"last update":"Wed Sep 18 10:33:18 2024","interfaces":[{"interface":"back","average":{"1min":1.2430000000000001,"5min":1.2430000000000001,"15min":1.2430000000000001},"min":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"max":{"1min":1.8180000000000001,"5min":1.8180000000000001,"15min":1.8180000000000001},"last":1.04},{"interface":"front","average":{"1min":1.276,"5min":1.276,"15min":1.276},"min":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"max":{"1min":1.681,"5min":1.681,"15min":1.681},"last":1.2170000000000001}]},{"osd":5,"last update":"Wed Sep 18 10:33:18 2024","interfaces":[{"interface":"back","average":{"1min":1.3140000000000001,"5min":1.3140000000000001,"15min":1.3140000000000001},"min":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"max":{"1min":1.8600000000000001,"5min":1.8600000000000001,"15min":1.8600000000000001},"last":1},{"interface":"front","average":{"1min":1.3009999999999999,"5min":1.3009999999999999,"15min":1.3009999999999999},"min":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"max":{"1min":1.7090000000000001,"5min":1.7090000000000001,"15min":1.7090000000000001},"last":1.3360000000000001}]},{"osd":6,"last update":"Wed Sep 18 10:33:18 2024","interfaces":[{"interface":"back","average":{"1min":1.3029999999999999,"5min":1.3029999999999999,"15min":1.3029999999999999},"min":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":2.778,"5min":2.778,"15min":2.778},"last":1.0700000000000001},{"interface":"front","average":{"1min":1.345,"5min":1.345,"15min":1.345},"min":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":1.921,"5min":1.921,"15min":1.921},"last":1.522}]},{"osd":7,"last update":"Wed Sep 18 10:33:18 2024","interfaces":[{"interface":"back","average":{"1min":1.3340000000000001,"5min":1.3340000000000001,"15min":1.3340000000000001},"min":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"max":{"1min":1.6830000000000001,"5min":1.6830000000000001,"15min":1.6830000000000001},"last":1.595},{"interface":"front","average":{"1min":1.417,"5min":1.417,"15min":1.417},"min":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"max":{"1min":3.0390000000000001,"5min":3.0390000000000001,"15min":3.0390000000000001},"last":1.121}]},{"osd":8,"last update":"Wed Sep 18 10:33:18 2024","interfaces":[{"interface":"back","average":{"1min":1.4330000000000001,"5min":1.4330000000000001,"15min":1.4330000000000001},"min":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"max":{"1min":2.8500000000000001,"5min":2.8500000000000001,"15min":2.8500000000000001},"last":1.5509999999999999},{"interface":"front","average":{"1min":1.4079999999999999,"5min":1.4079999999999999,"15min":1.4079999999999999},"min":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"max":{"1min":2.0070000000000001,"5min":2.0070000000000001,"15min":2.0070000000000001},"last":1.649}]}]},{"osd":8,"up_from":48,"seq":206158430233,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":104857600,"kb_used":298032,"kb_used_data":672,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":104559568,"statfs":{"total":107374182400,"available":107068997632,"internally_reserved":0,"allocated":688128,"data_stored":332771,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[2,3,4,5,6,7,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 10:33:16 2024","interfaces":[{"interface":"back","average":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":1.26,"5min":1.26,"15min":1.26},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":1.238,"5min":1.238,"15min":1.238},"last":0.86299999999999999}]},{"osd":1,"last update":"Wed Sep 18 10:33:16 2024","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.751,"15min":0.751},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":1.129,"5min":1.129,"15min":1.129},"last":1.1359999999999999},{"interface":"front","average":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"min":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.2390000000000001,"5min":1.2390000000000001,"15min":1.2390000000000001},"last":0.92600000000000005}]},{"osd":2,"last update":"Wed Sep 18 10:33:16 2024","interfaces":[{"interface":"back","average":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":1.6439999999999999,"5min":1.6439999999999999,"15min":1.6439999999999999},"last":0.98099999999999998},{"interface":"front","average":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":1.2529999999999999,"5min":1.2529999999999999,"15min":1.2529999999999999},"last":0.70999999999999996}]},{"osd":3,"last update":"Wed Sep 18 10:33:16 2024","interfaces":[{"interface":"back","average":{"1min":1.109,"5min":1.109,"15min":1.109},"min":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"max":{"1min":1.538,"5min":1.538,"15min":1.538},"last":0.78900000000000003},{"interface":"front","average":{"1min":1.0389999999999999,"5min":1.0389999999999999,"15min":1.0389999999999999},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.4810000000000001,"5min":1.4810000000000001,"15min":1.4810000000000001},"last":1.0549999999999999}]},{"osd":4,"last update":"Wed Sep 18 10:33:16 2024","interfaces":[{"interface":"back","average":{"1min":1.0620000000000001,"5min":1.0620000000000001,"15min":1.0620000000000001},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.8169999999999999,"5min":1.8169999999999999,"15min":1.8169999999999999},"last":1.1839999999999999},{"interface":"front","average":{"1min":1.2549999999999999,"5min":1.2549999999999999,"15min":1.2549999999999999},"min":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"max":{"1min":2.1480000000000001,"5min":2.1480000000000001,"15min":2.1480000000000001},"last":1.3220000000000001}]},{"osd":5,"last update":"Wed Sep 18 10:33:16 2024","interfaces":[{"interface":"back","average":{"1min":1.216,"5min":1.216,"15min":1.216},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":2.3109999999999999,"5min":2.3109999999999999,"15min":2.3109999999999999},"last":1.2190000000000001},{"interface":"front","average":{"1min":1.161,"5min":1.161,"15min":1.161},"min":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":2.0169999999999999,"5min":2.0169999999999999,"15min":2.0169999999999999},"last":1.363}]},{"osd":6,"last update":"Wed Sep 18 10:33:16 2024","interfaces":[{"interface":"back","average":{"1min":1.2889999999999999,"5min":1.2889999999999999,"15min":1.2889999999999999},"min":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"max":{"1min":2.1899999999999999,"5min":2.1899999999999999,"15min":2.1899999999999999},"last":1.5329999999999999},{"interface":"front","average":{"1min":1.1799999999999999,"5min":1.1799999999999999,"15min":1.1799999999999999},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":2.069,"5min":2.069,"15min":2.069},"last":1.2529999999999999}]},{"osd":7,"last update":"Wed Sep 18 10:33:16 2024","interfaces":[{"interface":"back","average":{"1min":1.2589999999999999,"5min":1.2589999999999999,"15min":1.2589999999999999},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":2.2130000000000001,"5min":2.2130000000000001,"15min":2.2130000000000001},"last":1.4399999999999999},{"interface":"front","average":{"1min":1.4039999999999999,"5min":1.4039999999999999,"15min":1.4039999999999999},"min":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"max":{"1min":2.2450000000000001,"5min":2.2450000000000001,"15min":2.2450000000000001},"last":1.472}]},{"osd":9,"last update":"Wed Sep 18 10:33:16 2024","interfaces":[{"interface":"back","average":{"1min":1.304,"5min":1.304,"15min":1.304},"min":{"1min":0.63,"5min":0.63,"15min":0.63},"max":{"1min":2.1800000000000002,"5min":2.1800000000000002,"15min":2.1800000000000002},"last":1.403},{"interface":"front","average":{"1min":1.3440000000000001,"5min":1.3440000000000001,"15min":1.3440000000000001},"min":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"max":{"1min":2.3969999999999998,"5min":2.3969999999999998,"15min":2.3969999999999998},"last":1.514}]}]},{"osd":7,"up_from":48,"seq":206158430232,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":298032,"kb_used_data":672,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":104559568,"statfs":{"total":107374182400,"available":107068997632,"internally_reserved":0,"allocated":688128,"data_stored":332771,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[2,3,4,5,6,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 10:33:17 2024","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.202,"5min":1.202,"15min":1.202},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.75,"5min":0.75,"15min":0.75},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":1.486,"5min":1.486,"15min":1.486},"last":0.98799999999999999}]},{"osd":1,"last update":"Wed Sep 18 10:33:17 2024","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.3009999999999999,"5min":1.3009999999999999,"15min":1.3009999999999999},"last":1.085},{"interface":"front","average":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.8480000000000001,"5min":1.8480000000000001,"15min":1.8480000000000001},"last":0.73799999999999999}]},{"osd":2,"last update":"Wed Sep 18 10:33:17 2024","interfaces":[{"interface":"back","average":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":1.5980000000000001,"5min":1.5980000000000001,"15min":1.5980000000000001},"last":1.036},{"interface":"front","average":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.3440000000000001,"5min":1.3440000000000001,"15min":1.3440000000000001},"last":1.212}]},{"osd":3,"last update":"Wed Sep 18 10:33:17 2024","interfaces":[{"interface":"back","average":{"1min":1.1000000000000001,"5min":1.1000000000000001,"15min":1.1000000000000001},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.9319999999999999,"5min":1.9319999999999999,"15min":1.9319999999999999},"last":0.91300000000000003},{"interface":"front","average":{"1min":1.0720000000000001,"5min":1.0720000000000001,"15min":1.0720000000000001},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.8879999999999999,"5min":1.8879999999999999,"15min":1.8879999999999999},"last":1.0600000000000001}]},{"osd":4,"last update":"Wed Sep 18 10:33:17 2024","interfaces":[{"interface":"back","average":{"1min":1.097,"5min":1.097,"15min":1.097},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":2.1379999999999999,"5min":2.1379999999999999,"15min":2.1379999999999999},"last":0.77600000000000002},{"interface":"front","average":{"1min":1.1060000000000001,"5min":1.1060000000000001,"15min":1.1060000000000001},"min":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":2.0110000000000001,"5min":2.0110000000000001,"15min":2.0110000000000001},"last":1.1639999999999999}]},{"osd":5,"last update":"Wed Sep 18 10:33:17 2024","interfaces":[{"interface":"back","average":{"1min":1.194,"5min":1.194,"15min":1.194},"min":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"max":{"1min":2.4889999999999999,"5min":2.4889999999999999,"15min":2.4889999999999999},"last":1.002},{"interface":"front","average":{"1min":1.2230000000000001,"5min":1.2230000000000001,"15min":1.2230000000000001},"min":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"max":{"1min":2.4289999999999998,"5min":2.4289999999999998,"15min":2.4289999999999998},"last":0.86499999999999999}]},{"osd":6,"last update":"Wed Sep 18 10:33:17 2024","interfaces":[{"interface":"back","average":{"1min":1.2769999999999999,"5min":1.2769999999999999,"15min":1.2769999999999999},"min":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"max":{"1min":2.0950000000000002,"5min":2.0950000000000002,"15min":2.0950000000000002},"last":1.284},{"interface":"front","average":{"1min":1.218,"5min":1.218,"15min":1.218},"min":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":1.9450000000000001,"5min":1.9450000000000001,"15min":1.9450000000000001},"last":1.1399999999999999}]},{"osd":8,"last update":"Wed Sep 18 10:33:17 2024","interfaces":[{"interface":"back","average":{"1min":1.2969999999999999,"5min":1.2969999999999999,"15min":1.2969999999999999},"min":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"max":{"1min":2.6389999999999998,"5min":2.6389999999999998,"15min":2.6389999999999998},"last":0.95999999999999996},{"interface":"front","average":{"1min":1.3600000000000001,"5min":1.3600000000000001,"15min":1.3600000000000001},"min":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"max":{"1min":2.1520000000000001,"5min":2.1520000000000001,"15min":2.1520000000000001},"last":1.5229999999999999}]},{"osd":9,"last update":"Wed Sep 18 10:33:17 2024","interfaces":[{"interface":"back","average":{"1min":1.2470000000000001,"5min":1.2470000000000001,"15min":1.2470000000000001},"min":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":2.0680000000000001,"5min":2.0680000000000001,"15min":2.0680000000000001},"last":1.6299999999999999},{"interface":"front","average":{"1min":1.282,"5min":1.282,"15min":1.282},"min":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"max":{"1min":1.8460000000000001,"5min":1.8460000000000001,"15min":1.8460000000000001},"last":1.1040000000000001}]}]},{"osd":6,"up_from":48,"seq":206158430232,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":298032,"kb_used_data":672,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":104559568,"statfs":{"total":107374182400,"available":107068997632,"internally_reserved":0,"allocated":688128,"data_stored":332771,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,7,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 10:33:17 2024","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":1.073,"5min":1.073,"15min":1.073},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":1.3720000000000001,"5min":1.3720000000000001,"15min":1.3720000000000001},"last":0.48499999999999999}]},{"osd":1,"last update":"Wed Sep 18 10:33:17 2024","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.751,"15min":0.751},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":1.6599999999999999,"5min":1.6599999999999999,"15min":1.6599999999999999},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":2.0459999999999998,"5min":2.0459999999999998,"15min":2.0459999999999998},"last":0.72399999999999998}]},{"osd":2,"last update":"Wed Sep 18 10:33:17 2024","interfaces":[{"interface":"back","average":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":1.1399999999999999,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":0.67400000000000004},{"interface":"front","average":{"1min":1.052,"5min":1.052,"15min":1.052},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":1.907,"5min":1.907,"15min":1.907},"last":0.84299999999999997}]},{"osd":3,"last update":"Wed Sep 18 10:33:17 2024","interfaces":[{"interface":"back","average":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.7490000000000001,"5min":1.7490000000000001,"15min":1.7490000000000001},"last":0.78600000000000003},{"interface":"front","average":{"1min":1.1830000000000001,"5min":1.1830000000000001,"15min":1.1830000000000001},"min":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"max":{"1min":1.8680000000000001,"5min":1.8680000000000001,"15min":1.8680000000000001},"last":1.0760000000000001}]},{"osd":4,"last update":"Wed Sep 18 10:33:17 2024","interfaces":[{"interface":"back","average":{"1min":1.2070000000000001,"5min":1.2070000000000001,"15min":1.2070000000000001},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.976,"5min":1.976,"15min":1.976},"last":1.153},{"interface":"front","average":{"1min":1.127,"5min":1.127,"15min":1.127},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":2.544,"5min":2.544,"15min":2.544},"last":1.427}]},{"osd":5,"last update":"Wed Sep 18 10:33:17 2024","interfaces":[{"interface":"back","average":{"1min":1.2170000000000001,"5min":1.2170000000000001,"15min":1.2170000000000001},"min":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"max":{"1min":2.1539999999999999,"5min":2.1539999999999999,"15min":2.1539999999999999},"last":1.1899999999999999},{"interface":"front","average":{"1min":1.296,"5min":1.296,"15min":1.296},"min":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":2.2469999999999999,"5min":2.2469999999999999,"15min":2.2469999999999999},"last":1.5509999999999999}]},{"osd":7,"last update":"Wed Sep 18 10:33:17 2024","interfaces":[{"interface":"back","average":{"1min":1.246,"5min":1.246,"15min":1.246},"min":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":2.6440000000000001,"5min":2.6440000000000001,"15min":2.6440000000000001},"last":1.044},{"interface":"front","average":{"1min":1.7609999999999999,"5min":1.7609999999999999,"15min":1.7609999999999999},"min":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"max":{"1min":10.385,"5min":10.385,"15min":10.385},"last":1.3240000000000001}]},{"osd":8,"last update":"Wed Sep 18 10:33:17 2024","interfaces":[{"interface":"back","average":{"1min":1.3640000000000001,"5min":1.3640000000000001,"15min":1.3640000000000001},"min":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"max":{"1min":2.444,"5min":2.444,"15min":2.444},"last":1.6180000000000001},{"interface":"front","average":{"1min":1.274,"5min":1.274,"15min":1.274},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":2.714,"5min":2.714,"15min":2.714},"last":1.1100000000000001}]},{"osd":9,"last update":"Wed Sep 18 10:33:17 2024","interfaces":[{"interface":"back","average":{"1min":1.2789999999999999,"5min":1.2789999999999999,"15min":1.2789999999999999},"min":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"max":{"1min":2.2879999999999998,"5min":2.2879999999999998,"15min":2.2879999999999998},"last":1.3620000000000001},{"interface":"front","average":{"1min":1.3240000000000001,"5min":1.3240000000000001,"15min":1.3240000000000001},"min":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":2.5960000000000001,"5min":2.5960000000000001,"15min":2.5960000000000001},"last":1.5860000000000001}]}]},{"osd":5,"up_from":48,"seq":206158430234,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":298028,"kb_used_data":672,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":104559572,"statfs":{"total":107374182400,"available":107069001728,"internally_reserved":0,"allocated":688128,"data_stored":332771,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,6,7,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 10:33:18 2024","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.111,"5min":1.111,"15min":1.111},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"min":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.5249999999999999,"5min":1.5249999999999999,"15min":1.5249999999999999},"last":0.67700000000000005}]},{"osd":1,"last update":"Wed Sep 18 10:33:18 2024","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.173,"5min":1.173,"15min":1.173},"last":1.02},{"interface":"front","average":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.8280000000000001,"5min":1.8280000000000001,"15min":1.8280000000000001},"last":1.1599999999999999}]},{"osd":2,"last update":"Wed Sep 18 10:33:18 2024","interfaces":[{"interface":"back","average":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"min":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":1.4159999999999999,"5min":1.4159999999999999,"15min":1.4159999999999999},"last":1.194},{"interface":"front","average":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"min":{"1min":0.627,"5min":0.627,"15min":0.627},"max":{"1min":2.3849999999999998,"5min":2.3849999999999998,"15min":2.3849999999999998},"last":1.0880000000000001}]},{"osd":3,"last update":"Wed Sep 18 10:33:18 2024","interfaces":[{"interface":"back","average":{"1min":1.079,"5min":1.079,"15min":1.079},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.923,"5min":1.923,"15min":1.923},"last":1.4139999999999999},{"interface":"front","average":{"1min":1.268,"5min":1.268,"15min":1.268},"min":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"max":{"1min":2.073,"5min":2.073,"15min":2.073},"last":1.5269999999999999}]},{"osd":4,"last update":"Wed Sep 18 10:33:18 2024","interfaces":[{"interface":"back","average":{"1min":1.123,"5min":1.123,"15min":1.123},"min":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"max":{"1min":2.141,"5min":2.141,"15min":2.141},"last":1.25},{"interface":"front","average":{"1min":1.2490000000000001,"5min":1.2490000000000001,"15min":1.2490000000000001},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":2.3300000000000001,"5min":2.3300000000000001,"15min":2.3300000000000001},"last":1.569}]},{"osd":6,"last update":"Wed Sep 18 10:33:18 2024","interfaces":[{"interface":"back","average":{"1min":1.2769999999999999,"5min":1.2769999999999999,"15min":1.2769999999999999},"min":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"max":{"1min":1.806,"5min":1.806,"15min":1.806},"last":1.2909999999999999},{"interface":"front","average":{"1min":1.274,"5min":1.274,"15min":1.274},"min":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"max":{"1min":2.6120000000000001,"5min":2.6120000000000001,"15min":2.6120000000000001},"last":1.05}]},{"osd":7,"last update":"Wed Sep 18 10:33:18 2024","interfaces":[{"interface":"back","average":{"1min":1.3380000000000001,"5min":1.3380000000000001,"15min":1.3380000000000001},"min":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"max":{"1min":2.661,"5min":2.661,"15min":2.661},"last":1.49},{"interface":"front","average":{"1min":1.3819999999999999,"5min":1.3819999999999999,"15min":1.3819999999999999},"min":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"max":{"1min":2.548,"5min":2.548,"15min":2.548},"last":1.3460000000000001}]},{"osd":8,"last update":"Wed Sep 18 10:33:18 2024","interfaces":[{"interface":"back","average":{"1min":1.359,"5min":1.359,"15min":1.359},"min":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"max":{"1min":2.7080000000000002,"5min":2.7080000000000002,"15min":2.7080000000000002},"last":1.4630000000000001},{"interface":"front","average":{"1min":1.327,"5min":1.327,"15min":1.327},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":2.4399999999999999,"5min":2.4399999999999999,"15min":2.4399999999999999},"last":1.645}]},{"osd":9,"last update":"Wed Sep 18 10:33:18 2024","interfaces":[{"interface":"back","average":{"1min":1.3859999999999999,"5min":1.3859999999999999,"15min":1.3859999999999999},"min":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"max":{"1min":2.2130000000000001,"5min":2.2130000000000001,"15min":2.2130000000000001},"last":1.1259999999999999},{"interface":"front","average":{"1min":1.4019999999999999,"5min":1.4019999999999999,"15min":1.4019999999999999},"min":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"max":{"1min":2.782,"5min":2.782,"15min":2.782},"last":1.3680000000000001}]}]},{"osd":4,"up_from":48,"seq":206158430232,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":298036,"kb_used_data":676,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":104559564,"statfs":{"total":107374182400,"available":107068993536,"internally_reserved":0,"allocated":692224,"data_stored":332790,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,5,6,7,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 10:33:16 2024","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":1.0960000000000001,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.745,"5min":0.745,"15min":0.745},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.171,"5min":1.171,"15min":1.171},"last":0.57499999999999996}]},{"osd":1,"last update":"Wed Sep 18 10:33:16 2024","interfaces":[{"interface":"back","average":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":1.1579999999999999,"5min":1.1579999999999999,"15min":1.1579999999999999},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.583,"5min":1.583,"15min":1.583},"last":1.038}]},{"osd":2,"last update":"Wed Sep 18 10:33:16 2024","interfaces":[{"interface":"back","average":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":2.1139999999999999,"5min":2.1139999999999999,"15min":2.1139999999999999},"last":1.0840000000000001},{"interface":"front","average":{"1min":0.995,"5min":0.995,"15min":0.995},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.5660000000000001,"5min":1.5660000000000001,"15min":1.5660000000000001},"last":1.302}]},{"osd":3,"last update":"Wed Sep 18 10:33:16 2024","interfaces":[{"interface":"back","average":{"1min":1.0620000000000001,"5min":1.0620000000000001,"15min":1.0620000000000001},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.6200000000000001,"5min":1.6200000000000001,"15min":1.6200000000000001},"last":0.84299999999999997},{"interface":"front","average":{"1min":1.1699999999999999,"5min":1.1699999999999999,"15min":1.1699999999999999},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.732,"5min":1.732,"15min":1.732},"last":1.337}]},{"osd":5,"last update":"Wed Sep 18 10:33:16 2024","interfaces":[{"interface":"back","average":{"1min":1.159,"5min":1.159,"15min":1.159},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":2.2509999999999999,"5min":2.2509999999999999,"15min":2.2509999999999999},"last":1.421},{"interface":"front","average":{"1min":1.1830000000000001,"5min":1.1830000000000001,"15min":1.1830000000000001},"min":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":1.873,"5min":1.873,"15min":1.873},"last":1.2070000000000001}]},{"osd":6,"last update":"Wed Sep 18 10:33:16 2024","interfaces":[{"interface":"back","average":{"1min":1.214,"5min":1.214,"15min":1.214},"min":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"max":{"1min":2.2879999999999998,"5min":2.2879999999999998,"15min":2.2879999999999998},"last":0.89500000000000002},{"interface":"front","average":{"1min":1.25,"5min":1.25,"15min":1.25},"min":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":2.661,"5min":2.661,"15min":2.661},"last":2.0750000000000002}]},{"osd":7,"last update":"Wed Sep 18 10:33:16 2024","interfaces":[{"interface":"back","average":{"1min":1.319,"5min":1.319,"15min":1.319},"min":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"max":{"1min":1.895,"5min":1.895,"15min":1.895},"last":2.7919999999999998},{"interface":"front","average":{"1min":1.2450000000000001,"5min":1.2450000000000001,"15min":1.2450000000000001},"min":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":2.0579999999999998,"5min":2.0579999999999998,"15min":2.0579999999999998},"last":1.5649999999999999}]},{"osd":8,"last update":"Wed Sep 18 10:33:16 2024","interfaces":[{"interface":"back","average":{"1min":1.278,"5min":1.278,"15min":1.278},"min":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":2.173,"5min":2.173,"15min":2.173},"last":1.3720000000000001},{"interface":"front","average":{"1min":1.3240000000000001,"5min":1.3240000000000001,"15min":1.3240000000000001},"min":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":1.9750000000000001,"5min":1.9750000000000001,"15min":1.9750000000000001},"last":1.7949999999999999}]},{"osd":9,"last update":"Wed Sep 18 10:33:16 2024","interfaces":[{"interface":"back","average":{"1min":1.327,"5min":1.327,"15min":1.327},"min":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":2.448,"5min":2.448,"15min":2.448},"last":1.704},{"interface":"front","average":{"1min":1.389,"5min":1.389,"15min":1.389},"min":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"max":{"1min":2.242,"5min":2.242,"15min":2.242},"last":2.1619999999999999}]}]},{"osd":3,"up_from":48,"seq":206158430234,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":298032,"kb_used_data":672,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":104559568,"statfs":{"total":107374182400,"available":107068997632,"internally_reserved":0,"allocated":688128,"data_stored":332771,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,4,5,6,7,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 10:33:15 2024","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.095,"5min":1.095,"15min":1.095},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"min":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":2.121,"5min":2.121,"15min":2.121},"last":1.2450000000000001}]},{"osd":1,"last update":"Wed Sep 18 10:33:15 2024","interfaces":[{"interface":"back","average":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":1.9259999999999999,"5min":1.9259999999999999,"15min":1.9259999999999999},"last":0.83099999999999996},{"interface":"front","average":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"min":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":1.2090000000000001,"5min":1.2090000000000001,"15min":1.2090000000000001},"last":1.3819999999999999}]},{"osd":2,"last update":"Wed Sep 18 10:33:15 2024","interfaces":[{"interface":"back","average":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.204,"5min":1.204,"15min":1.204},"last":0.624},{"interface":"front","average":{"1min":1.1870000000000001,"5min":1.1870000000000001,"15min":1.1870000000000001},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":3.2040000000000002,"5min":3.2040000000000002,"15min":3.2040000000000002},"last":0.59699999999999998}]},{"osd":4,"last update":"Wed Sep 18 10:33:15 2024","interfaces":[{"interface":"back","average":{"1min":1.071,"5min":1.071,"15min":1.071},"min":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":1.49,"5min":1.49,"15min":1.49},"last":0.75900000000000001},{"interface":"front","average":{"1min":1.266,"5min":1.266,"15min":1.266},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":2.7519999999999998,"5min":2.7519999999999998,"15min":2.7519999999999998},"last":1.3200000000000001}]},{"osd":5,"last update":"Wed Sep 18 10:33:15 2024","interfaces":[{"interface":"back","average":{"1min":1.1830000000000001,"5min":1.1830000000000001,"15min":1.1830000000000001},"min":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"max":{"1min":2.0630000000000002,"5min":2.0630000000000002,"15min":2.0630000000000002},"last":1.145},{"interface":"front","average":{"1min":1.1259999999999999,"5min":1.1259999999999999,"15min":1.1259999999999999},"min":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"max":{"1min":1.5149999999999999,"5min":1.5149999999999999,"15min":1.5149999999999999},"last":0.71699999999999997}]},{"osd":6,"last update":"Wed Sep 18 10:33:15 2024","interfaces":[{"interface":"back","average":{"1min":1.3620000000000001,"5min":1.3620000000000001,"15min":1.3620000000000001},"min":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":4.1029999999999998,"5min":4.1029999999999998,"15min":4.1029999999999998},"last":1.0389999999999999},{"interface":"front","average":{"1min":1.268,"5min":1.268,"15min":1.268},"min":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"max":{"1min":2.2789999999999999,"5min":2.2789999999999999,"15min":2.2789999999999999},"last":0.95399999999999996}]},{"osd":7,"last update":"Wed Sep 18 10:33:15 2024","interfaces":[{"interface":"back","average":{"1min":1.2010000000000001,"5min":1.2010000000000001,"15min":1.2010000000000001},"min":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"max":{"1min":2.1880000000000002,"5min":2.1880000000000002,"15min":2.1880000000000002},"last":0.89000000000000001},{"interface":"front","average":{"1min":1.282,"5min":1.282,"15min":1.282},"min":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"max":{"1min":3.0409999999999999,"5min":3.0409999999999999,"15min":3.0409999999999999},"last":1.429}]},{"osd":8,"last update":"Wed Sep 18 10:33:15 2024","interfaces":[{"interface":"back","average":{"1min":1.5109999999999999,"5min":1.5109999999999999,"15min":1.5109999999999999},"min":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"max":{"1min":3.5590000000000002,"5min":3.5590000000000002,"15min":3.5590000000000002},"last":1.476},{"interface":"front","average":{"1min":1.3540000000000001,"5min":1.3540000000000001,"15min":1.3540000000000001},"min":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"max":{"1min":2.3919999999999999,"5min":2.3919999999999999,"15min":2.3919999999999999},"last":1.3600000000000001}]},{"osd":9,"last update":"Wed Sep 18 10:33:15 2024","interfaces":[{"interface":"back","average":{"1min":1.379,"5min":1.379,"15min":1.379},"min":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"max":{"1min":3.1259999999999999,"5min":3.1259999999999999,"15min":3.1259999999999999},"last":1.5169999999999999},{"interface":"front","average":{"1min":1.5109999999999999,"5min":1.5109999999999999,"15min":1.5109999999999999},"min":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"max":{"1min":3.6339999999999999,"5min":3.6339999999999999,"15min":3.6339999999999999},"last":1.292}]}]},{"osd":2,"up_from":48,"seq":206158430232,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":104857600,"kb_used":298028,"kb_used_data":672,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":104559572,"statfs":{"total":107374182400,"available":107069001728,"internally_reserved":0,"allocated":688128,"data_stored":332771,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,3,4,5,6,7,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 10:33:18 2024","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1.109,"5min":1.109,"15min":1.109},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":1.5029999999999999,"5min":1.5029999999999999,"15min":1.5029999999999999},"last":1.119}]},{"osd":1,"last update":"Wed Sep 18 10:33:18 2024","interfaces":[{"interface":"back","average":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"min":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.651,"5min":1.651,"15min":1.651},"last":0.82599999999999996},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":1.1559999999999999,"5min":1.1559999999999999,"15min":1.1559999999999999},"last":1.2769999999999999}]},{"osd":3,"last update":"Wed Sep 18 10:33:18 2024","interfaces":[{"interface":"back","average":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":1.2869999999999999,"5min":1.2869999999999999,"15min":1.2869999999999999},"last":0.92200000000000004},{"interface":"front","average":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.7390000000000001,"5min":1.7390000000000001,"15min":1.7390000000000001},"last":0.64100000000000001}]},{"osd":4,"last update":"Wed Sep 18 10:33:18 2024","interfaces":[{"interface":"back","average":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"min":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":1.333,"5min":1.333,"15min":1.333},"last":1.3540000000000001},{"interface":"front","average":{"1min":1.151,"5min":1.151,"15min":1.151},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.7869999999999999,"5min":1.7869999999999999,"15min":1.7869999999999999},"last":1.032}]},{"osd":5,"last update":"Wed Sep 18 10:33:18 2024","interfaces":[{"interface":"back","average":{"1min":1.1100000000000001,"5min":1.1100000000000001,"15min":1.1100000000000001},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.597,"5min":1.597,"15min":1.597},"last":1.417},{"interface":"front","average":{"1min":1.246,"5min":1.246,"15min":1.246},"min":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"max":{"1min":1.873,"5min":1.873,"15min":1.873},"last":1.159}]},{"osd":6,"last update":"Wed Sep 18 10:33:18 2024","interfaces":[{"interface":"back","average":{"1min":1.0649999999999999,"5min":1.0649999999999999,"15min":1.0649999999999999},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":1.696,"5min":1.696,"15min":1.696},"last":1.093},{"interface":"front","average":{"1min":1.1479999999999999,"5min":1.1479999999999999,"15min":1.1479999999999999},"min":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"max":{"1min":1.5289999999999999,"5min":1.5289999999999999,"15min":1.5289999999999999},"last":1.8899999999999999}]},{"osd":7,"last update":"Wed Sep 18 10:33:18 2024","interfaces":[{"interface":"back","average":{"1min":1.268,"5min":1.268,"15min":1.268},"min":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":1.9350000000000001,"5min":1.9350000000000001,"15min":1.9350000000000001},"last":0.97299999999999998},{"interface":"front","average":{"1min":1.244,"5min":1.244,"15min":1.244},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":2.6419999999999999,"5min":2.6419999999999999,"15min":2.6419999999999999},"last":1.8069999999999999}]},{"osd":8,"last update":"Wed Sep 18 10:33:18 2024","interfaces":[{"interface":"back","average":{"1min":1.306,"5min":1.306,"15min":1.306},"min":{"1min":1,"5min":1,"15min":1},"max":{"1min":1.613,"5min":1.613,"15min":1.613},"last":1.3160000000000001},{"interface":"front","average":{"1min":1.278,"5min":1.278,"15min":1.278},"min":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":1.9830000000000001,"5min":1.9830000000000001,"15min":1.9830000000000001},"last":1.3779999999999999}]},{"osd":9,"last update":"Wed Sep 18 10:33:18 2024","interfaces":[{"interface":"back","average":{"1min":1.2210000000000001,"5min":1.2210000000000001,"15min":1.2210000000000001},"min":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"max":{"1min":2.4220000000000002,"5min":2.4220000000000002,"15min":2.4220000000000002},"last":2.0699999999999998},{"interface":"front","average":{"1min":1.337,"5min":1.337,"15min":1.337},"min":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"max":{"1min":2.1349999999999998,"5min":2.1349999999999998,"15min":2.1349999999999998},"last":1.24}]}]},{"osd":1,"up_from":59,"seq":253403070489,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":298040,"kb_used_data":676,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":104559560,"statfs":{"total":107374182400,"available":107068989440,"internally_reserved":0,"allocated":692224,"data_stored":332790,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,2,3,4,5,6,7,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 10:33:17 2024","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":1.0980000000000001,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":1.3979999999999999},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"min":{"1min":0.248,"5min":0.248,"15min":0.248},"max":{"1min":1.3,"5min":1.3,"15min":1.3},"last":0.85299999999999998}]},{"osd":2,"last update":"Wed Sep 18 10:33:17 2024","interfaces":[{"interface":"back","average":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.206,"5min":1.206,"15min":1.206},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":1.54,"5min":1.54,"15min":1.54},"last":2.23}]},{"osd":3,"last update":"Wed Sep 18 10:33:17 2024","interfaces":[{"interface":"back","average":{"1min":1.129,"5min":1.129,"15min":1.129},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":5.7569999999999997,"5min":5.7569999999999997,"15min":5.7569999999999997},"last":0.94299999999999995},{"interface":"front","average":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":2.9830000000000001,"5min":2.9830000000000001,"15min":2.9830000000000001},"last":0.70199999999999996}]},{"osd":4,"last update":"Wed Sep 18 10:33:17 2024","interfaces":[{"interface":"back","average":{"1min":1.1000000000000001,"5min":1.1000000000000001,"15min":1.1000000000000001},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":3.8700000000000001,"5min":3.8700000000000001,"15min":3.8700000000000001},"last":0.73399999999999999},{"interface":"front","average":{"1min":1.1359999999999999,"5min":1.1359999999999999,"15min":1.1359999999999999},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":2.6520000000000001,"5min":2.6520000000000001,"15min":2.6520000000000001},"last":2.0670000000000002}]},{"osd":5,"last update":"Wed Sep 18 10:33:17 2024","interfaces":[{"interface":"back","average":{"1min":1.1160000000000001,"5min":1.1160000000000001,"15min":1.1160000000000001},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":2.0470000000000002,"5min":2.0470000000000002,"15min":2.0470000000000002},"last":2.1560000000000001},{"interface":"front","average":{"1min":1.0920000000000001,"5min":1.0920000000000001,"15min":1.0920000000000001},"min":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"max":{"1min":1.6679999999999999,"5min":1.6679999999999999,"15min":1.6679999999999999},"last":1.0409999999999999}]},{"osd":6,"last update":"Wed Sep 18 10:33:17 2024","interfaces":[{"interface":"back","average":{"1min":1.3280000000000001,"5min":1.3280000000000001,"15min":1.3280000000000001},"min":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":3.1299999999999999,"5min":3.1299999999999999,"15min":3.1299999999999999},"last":1.125},{"interface":"front","average":{"1min":1.123,"5min":1.123,"15min":1.123},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.9390000000000001,"5min":1.9390000000000001,"15min":1.9390000000000001},"last":0.97799999999999998}]},{"osd":7,"last update":"Wed Sep 18 10:33:17 2024","interfaces":[{"interface":"back","average":{"1min":1.1060000000000001,"5min":1.1060000000000001,"15min":1.1060000000000001},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":2.4900000000000002,"5min":2.4900000000000002,"15min":2.4900000000000002},"last":1.256},{"interface":"front","average":{"1min":1.3120000000000001,"5min":1.3120000000000001,"15min":1.3120000000000001},"min":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":3.2810000000000001,"5min":3.2810000000000001,"15min":3.2810000000000001},"last":2.3010000000000002}]},{"osd":8,"last update":"Wed Sep 18 10:33:17 2024","interfaces":[{"interface":"back","average":{"1min":1.258,"5min":1.258,"15min":1.258},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":4.4969999999999999,"5min":4.4969999999999999,"15min":4.4969999999999999},"last":1.3089999999999999},{"interface":"front","average":{"1min":1.1419999999999999,"5min":1.1419999999999999,"15min":1.1419999999999999},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":2.1629999999999998,"5min":2.1629999999999998,"15min":2.1629999999999998},"last":1.169}]},{"osd":9,"last update":"Wed Sep 18 10:33:17 2024","interfaces":[{"interface":"back","average":{"1min":1.4419999999999999,"5min":1.4419999999999999,"15min":1.4419999999999999},"min":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"max":{"1min":3.2160000000000002,"5min":3.2160000000000002,"15min":3.2160000000000002},"last":2.3799999999999999},{"interface":"front","average":{"1min":1.2410000000000001,"5min":1.2410000000000001,"15min":1.2410000000000001},"min":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"max":{"1min":1.9430000000000001,"5min":1.9430000000000001,"15min":1.9430000000000001},"last":1.202}]}]},{"osd":0,"up_from":59,"seq":253403070489,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":298036,"kb_used_data":672,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":104559564,"statfs":{"total":107374182400,"available":107068993536,"internally_reserved":0,"allocated":688128,"data_stored":332771,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[1,2,3,4,5,6,7,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Sep 18 10:33:15 2024","interfaces":[{"interface":"back","average":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":1.5880000000000001,"5min":1.5880000000000001,"15min":1.5880000000000001},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.917,"5min":1.917,"15min":1.917},"last":2.218}]},{"osd":2,"last update":"Wed Sep 18 10:33:15 2024","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.746,"15min":0.746},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.454,"5min":1.454,"15min":1.454},"last":1.5409999999999999},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"min":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.222,"5min":1.222,"15min":1.222},"last":0.57399999999999995}]},{"osd":3,"last update":"Wed Sep 18 10:33:15 2024","interfaces":[{"interface":"back","average":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":1.5509999999999999,"5min":1.5509999999999999,"15min":1.5509999999999999},"last":1.0589999999999999},{"interface":"front","average":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.6339999999999999,"5min":1.6339999999999999,"15min":1.6339999999999999},"last":2.137}]},{"osd":4,"last update":"Wed Sep 18 10:33:15 2024","interfaces":[{"interface":"back","average":{"1min":1,"5min":1,"15min":1},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":1.6339999999999999,"5min":1.6339999999999999,"15min":1.6339999999999999},"last":0.91000000000000003},{"interface":"front","average":{"1min":1.093,"5min":1.093,"15min":1.093},"min":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.704,"5min":1.704,"15min":1.704},"last":1.1399999999999999}]},{"osd":5,"last update":"Wed Sep 18 10:33:15 2024","interfaces":[{"interface":"back","average":{"1min":1.1699999999999999,"5min":1.1699999999999999,"15min":1.1699999999999999},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.9590000000000001,"5min":1.9590000000000001,"15min":1.9590000000000001},"last":2.2829999999999999},{"interface":"front","average":{"1min":1.2050000000000001,"5min":1.2050000000000001,"15min":1.2050000000000001},"min":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"max":{"1min":2.206,"5min":2.206,"15min":2.206},"last":1.4330000000000001}]},{"osd":6,"last update":"Wed Sep 18 10:33:15 2024","interfaces":[{"interface":"back","average":{"1min":1.157,"5min":1.157,"15min":1.157},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":1.74,"5min":1.74,"15min":1.74},"last":1.974},{"interface":"front","average":{"1min":1.135,"5min":1.135,"15min":1.135},"min":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"max":{"1min":2.8140000000000001,"5min":2.8140000000000001,"15min":2.8140000000000001},"last":0.998}]},{"osd":7,"last update":"Wed Sep 18 10:33:15 2024","interfaces":[{"interface":"back","average":{"1min":1.268,"5min":1.268,"15min":1.268},"min":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"max":{"1min":2.004,"5min":2.004,"15min":2.004},"last":1.673},{"interface":"front","average":{"1min":1.357,"5min":1.357,"15min":1.357},"min":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"max":{"1min":2.7250000000000001,"5min":2.7250000000000001,"15min":2.7250000000000001},"last":2.0630000000000002}]},{"osd":8,"last update":"Wed Sep 18 10:33:15 2024","interfaces":[{"interface":"back","average":{"1min":1.3049999999999999,"5min":1.3049999999999999,"15min":1.3049999999999999},"min":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":2.1419999999999999,"5min":2.1419999999999999,"15min":2.1419999999999999},"last":1.837},{"interface":"front","average":{"1min":1.3779999999999999,"5min":1.3779999999999999,"15min":1.3779999999999999},"min":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"max":{"1min":2.8980000000000001,"5min":2.8980000000000001,"15min":2.8980000000000001},"last":2.6379999999999999}]},{"osd":9,"last update":"Wed Sep 18 10:33:15 2024","interfaces":[{"interface":"back","average":{"1min":1.2989999999999999,"5min":1.2989999999999999,"15min":1.2989999999999999},"min":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"max":{"1min":2.3109999999999999,"5min":2.3109999999999999,"15min":2.3109999999999999},"last":1.1080000000000001},{"interface":"front","average":{"1min":1.333,"5min":1.333,"15min":1.333},"min":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":2.2639999999999998,"5min":2.2639999999999998,"15min":2.2639999999999998},"last":1.198}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-18T10:34:08.540 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-18T10:34:08.815 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-18T10:34:08.816 INFO:teuthology.orchestra.run.smithi134.stdout:{"epoch":60,"fsid":"909a0a21-79ad-49e6-bd09-2f76d1ea12ab","created":"2024-09-18T10:32:06.815161+0000","modified":"2024-09-18T10:34:03.126450+0000","last_up_change":"2024-09-18T10:34:02.112750+0000","last_in_change":"2024-09-18T10:32:09.430449+0000","flags":"noout,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5800064,"flags_set":["noout","pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":10,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":10,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2024-09-18T10:32:18.531989+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":20,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"5d600f22-6bed-4106-a48d-b8af006ace47","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":15,"last_clean_end":58,"up_from":59,"up_thru":38,"down_at":55,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6834","nonce":25641},{"type":"v1","addr":"172.21.15.134:6837","nonce":25641}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6843","nonce":3025641},{"type":"v1","addr":"172.21.15.134:6859","nonce":3025641}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6853","nonce":25641},{"type":"v1","addr":"172.21.15.134:6855","nonce":25641}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6847","nonce":25641},{"type":"v1","addr":"172.21.15.134:6850","nonce":25641}]},"public_addr":"172.21.15.134:6837/25641","cluster_addr":"172.21.15.134:6859/3025641","heartbeat_back_addr":"172.21.15.134:6855/25641","heartbeat_front_addr":"172.21.15.134:6850/25641","state":["exists","up"]},{"osd":1,"uuid":"17de89fa-8f8b-495e-8fd6-24d62bfbb4de","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":15,"last_clean_end":58,"up_from":59,"up_thru":59,"down_at":55,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6810","nonce":25636},{"type":"v1","addr":"172.21.15.134:6813","nonce":25636}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6816","nonce":3025636},{"type":"v1","addr":"172.21.15.134:6818","nonce":3025636}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6827","nonce":25636},{"type":"v1","addr":"172.21.15.134:6830","nonce":25636}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6822","nonce":25636},{"type":"v1","addr":"172.21.15.134:6825","nonce":25636}]},"public_addr":"172.21.15.134:6813/25636","cluster_addr":"172.21.15.134:6818/3025636","heartbeat_back_addr":"172.21.15.134:6830/25636","heartbeat_front_addr":"172.21.15.134:6825/25636","state":["exists","up"]},{"osd":2,"uuid":"19059e59-5e31-4f29-83a3-9b068d17b228","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":15,"last_clean_end":47,"up_from":48,"up_thru":0,"down_at":44,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6800","nonce":25637},{"type":"v1","addr":"172.21.15.134:6801","nonce":25637}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6869","nonce":1025637},{"type":"v1","addr":"172.21.15.134:6871","nonce":1025637}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6806","nonce":25637},{"type":"v1","addr":"172.21.15.134:6807","nonce":25637}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6804","nonce":25637},{"type":"v1","addr":"172.21.15.134:6805","nonce":25637}]},"public_addr":"172.21.15.134:6801/25637","cluster_addr":"172.21.15.134:6871/1025637","heartbeat_back_addr":"172.21.15.134:6807/25637","heartbeat_front_addr":"172.21.15.134:6805/25637","state":["exists","up"]},{"osd":3,"uuid":"636276c9-3bda-4382-95c1-03bb98e80ac3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":15,"last_clean_end":47,"up_from":48,"up_thru":59,"down_at":44,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6812","nonce":25639},{"type":"v1","addr":"172.21.15.134:6814","nonce":25639}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6802","nonce":1025639},{"type":"v1","addr":"172.21.15.134:6803","nonce":1025639}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6826","nonce":25639},{"type":"v1","addr":"172.21.15.134:6829","nonce":25639}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6821","nonce":25639},{"type":"v1","addr":"172.21.15.134:6824","nonce":25639}]},"public_addr":"172.21.15.134:6814/25639","cluster_addr":"172.21.15.134:6803/1025639","heartbeat_back_addr":"172.21.15.134:6829/25639","heartbeat_front_addr":"172.21.15.134:6824/25639","state":["exists","up"]},{"osd":4,"uuid":"b918785a-5612-49ab-9b71-78433ef817f0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":15,"last_clean_end":47,"up_from":48,"up_thru":59,"down_at":44,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6808","nonce":25640},{"type":"v1","addr":"172.21.15.134:6809","nonce":25640}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6882","nonce":1025640},{"type":"v1","addr":"172.21.15.134:6883","nonce":1025640}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6828","nonce":25640},{"type":"v1","addr":"172.21.15.134:6831","nonce":25640}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6819","nonce":25640},{"type":"v1","addr":"172.21.15.134:6823","nonce":25640}]},"public_addr":"172.21.15.134:6809/25640","cluster_addr":"172.21.15.134:6883/1025640","heartbeat_back_addr":"172.21.15.134:6831/25640","heartbeat_front_addr":"172.21.15.134:6823/25640","state":["exists","up"]},{"osd":5,"uuid":"4b051f94-3f16-4114-83dc-2ea21f6da775","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":15,"last_clean_end":47,"up_from":48,"up_thru":48,"down_at":44,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6832","nonce":25645},{"type":"v1","addr":"172.21.15.134:6835","nonce":25645}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6811","nonce":1025645},{"type":"v1","addr":"172.21.15.134:6815","nonce":1025645}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6849","nonce":25645},{"type":"v1","addr":"172.21.15.134:6851","nonce":25645}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6844","nonce":25645},{"type":"v1","addr":"172.21.15.134:6846","nonce":25645}]},"public_addr":"172.21.15.134:6835/25645","cluster_addr":"172.21.15.134:6815/1025645","heartbeat_back_addr":"172.21.15.134:6851/25645","heartbeat_front_addr":"172.21.15.134:6846/25645","state":["exists","up"]},{"osd":6,"uuid":"7771b052-b691-4ede-9703-fa5a4bc2c6e2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":15,"last_clean_end":47,"up_from":48,"up_thru":55,"down_at":44,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6856","nonce":25767},{"type":"v1","addr":"172.21.15.134:6857","nonce":25767}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6868","nonce":1025767},{"type":"v1","addr":"172.21.15.134:6870","nonce":1025767}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6862","nonce":25767},{"type":"v1","addr":"172.21.15.134:6863","nonce":25767}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6860","nonce":25767},{"type":"v1","addr":"172.21.15.134:6861","nonce":25767}]},"public_addr":"172.21.15.134:6857/25767","cluster_addr":"172.21.15.134:6870/1025767","heartbeat_back_addr":"172.21.15.134:6863/25767","heartbeat_front_addr":"172.21.15.134:6861/25767","state":["exists","up"]},{"osd":7,"uuid":"e956d2c2-b6e6-422f-a052-5e7efe34ce26","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":15,"last_clean_end":47,"up_from":48,"up_thru":48,"down_at":44,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6864","nonce":25642},{"type":"v1","addr":"172.21.15.134:6866","nonce":25642}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6838","nonce":1025642},{"type":"v1","addr":"172.21.15.134:6841","nonce":1025642}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6876","nonce":25642},{"type":"v1","addr":"172.21.15.134:6878","nonce":25642}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6872","nonce":25642},{"type":"v1","addr":"172.21.15.134:6874","nonce":25642}]},"public_addr":"172.21.15.134:6866/25642","cluster_addr":"172.21.15.134:6841/1025642","heartbeat_back_addr":"172.21.15.134:6878/25642","heartbeat_front_addr":"172.21.15.134:6874/25642","state":["exists","up"]},{"osd":8,"uuid":"1896fcc5-d899-4784-8a2c-11344b1860e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":15,"last_clean_end":47,"up_from":48,"up_thru":17,"down_at":44,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6865","nonce":25694},{"type":"v1","addr":"172.21.15.134:6867","nonce":25694}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6858","nonce":2025694},{"type":"v1","addr":"172.21.15.134:6840","nonce":2025694}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6877","nonce":25694},{"type":"v1","addr":"172.21.15.134:6879","nonce":25694}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6873","nonce":25694},{"type":"v1","addr":"172.21.15.134:6875","nonce":25694}]},"public_addr":"172.21.15.134:6867/25694","cluster_addr":"172.21.15.134:6840/2025694","heartbeat_back_addr":"172.21.15.134:6879/25694","heartbeat_front_addr":"172.21.15.134:6875/25694","state":["exists","up"]},{"osd":9,"uuid":"818002b3-6bbd-4802-95e5-eaef0a009ccb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":15,"last_clean_end":40,"up_from":41,"up_thru":59,"down_at":38,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6833","nonce":25649},{"type":"v1","addr":"172.21.15.134:6836","nonce":25649}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6817","nonce":1025649},{"type":"v1","addr":"172.21.15.134:6820","nonce":1025649}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6852","nonce":25649},{"type":"v1","addr":"172.21.15.134:6854","nonce":25649}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.134:6845","nonce":25649},{"type":"v1","addr":"172.21.15.134:6848","nonce":25649}]},"public_addr":"172.21.15.134:6836/25649","cluster_addr":"172.21.15.134:6820/1025649","heartbeat_back_addr":"172.21.15.134:6854/25649","heartbeat_front_addr":"172.21.15.134:6848/25649","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"2024-09-18T10:33:58.088354+0000","laggy_probability":0.65700000524520874,"laggy_interval":1,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T10:32:13.629735+0000","dead_epoch":56},{"osd":1,"down_stamp":"2024-09-18T10:33:58.088354+0000","laggy_probability":0.65700000524520874,"laggy_interval":1,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T10:32:13.634602+0000","dead_epoch":57},{"osd":2,"down_stamp":"2024-09-18T10:33:46.992600+0000","laggy_probability":0.30000001192092896,"laggy_interval":1,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T10:32:13.644041+0000","dead_epoch":44},{"osd":3,"down_stamp":"2024-09-18T10:33:46.992600+0000","laggy_probability":0.30000001192092896,"laggy_interval":1,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T10:32:13.431697+0000","dead_epoch":44},{"osd":4,"down_stamp":"2024-09-18T10:33:46.992600+0000","laggy_probability":0.30000001192092896,"laggy_interval":1,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T10:32:13.848559+0000","dead_epoch":44},{"osd":5,"down_stamp":"2024-09-18T10:33:46.992600+0000","laggy_probability":0.30000001192092896,"laggy_interval":1,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T10:32:13.632248+0000","dead_epoch":44},{"osd":6,"down_stamp":"2024-09-18T10:33:46.992600+0000","laggy_probability":0.30000001192092896,"laggy_interval":1,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T10:32:13.676354+0000","dead_epoch":45},{"osd":7,"down_stamp":"2024-09-18T10:33:46.992600+0000","laggy_probability":0.30000001192092896,"laggy_interval":1,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T10:32:14.111210+0000","dead_epoch":44},{"osd":8,"down_stamp":"2024-09-18T10:33:46.992600+0000","laggy_probability":0.50999999046325684,"laggy_interval":1,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T10:32:14.173467+0000","dead_epoch":47},{"osd":9,"down_stamp":"2024-09-18T10:33:38.944822+0000","laggy_probability":0.30000001192092896,"laggy_interval":1,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-18T10:32:13.436859+0000","dead_epoch":38}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-18T10:34:09.830 INFO:tasks.ceph:Scrubbing osd.0 2024-09-18T10:34:09.831 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-09-18T10:34:09.977 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-18T10:34:09.978 INFO:teuthology.orchestra.run.smithi134.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-18T10:34:09.978 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-18T10:34:09.991 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-09-18T10:34:10.292 INFO:teuthology.orchestra.run.smithi134.stderr:instructed osd(s) 0 to deep-scrub 2024-09-18T10:34:10.302 INFO:tasks.ceph:Scrubbing osd.1 2024-09-18T10:34:10.302 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-09-18T10:34:10.444 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-18T10:34:10.444 INFO:teuthology.orchestra.run.smithi134.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-18T10:34:10.444 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-18T10:34:10.455 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-09-18T10:34:10.747 INFO:teuthology.orchestra.run.smithi134.stderr:instructed osd(s) 1 to deep-scrub 2024-09-18T10:34:10.762 INFO:tasks.ceph:Scrubbing osd.2 2024-09-18T10:34:10.762 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-09-18T10:34:10.898 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-18T10:34:10.899 INFO:teuthology.orchestra.run.smithi134.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-18T10:34:10.899 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-18T10:34:10.911 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-09-18T10:34:11.210 INFO:teuthology.orchestra.run.smithi134.stderr:instructed osd(s) 2 to deep-scrub 2024-09-18T10:34:11.225 INFO:tasks.ceph:Scrubbing osd.3 2024-09-18T10:34:11.225 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2024-09-18T10:34:11.361 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-18T10:34:11.361 INFO:teuthology.orchestra.run.smithi134.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-18T10:34:11.361 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-18T10:34:11.374 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2024-09-18T10:34:11.664 INFO:teuthology.orchestra.run.smithi134.stderr:instructed osd(s) 3 to deep-scrub 2024-09-18T10:34:11.673 INFO:tasks.ceph:Scrubbing osd.4 2024-09-18T10:34:11.674 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2024-09-18T10:34:11.807 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-18T10:34:11.807 INFO:teuthology.orchestra.run.smithi134.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-18T10:34:11.807 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-18T10:34:11.819 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2024-09-18T10:34:12.116 INFO:teuthology.orchestra.run.smithi134.stderr:instructed osd(s) 4 to deep-scrub 2024-09-18T10:34:12.126 INFO:tasks.ceph:Scrubbing osd.5 2024-09-18T10:34:12.126 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2024-09-18T10:34:12.273 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-18T10:34:12.273 INFO:teuthology.orchestra.run.smithi134.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-18T10:34:12.273 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-18T10:34:12.285 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2024-09-18T10:34:12.579 INFO:teuthology.orchestra.run.smithi134.stderr:instructed osd(s) 5 to deep-scrub 2024-09-18T10:34:12.589 INFO:tasks.ceph:Scrubbing osd.6 2024-09-18T10:34:12.590 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2024-09-18T10:34:12.736 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-18T10:34:12.737 INFO:teuthology.orchestra.run.smithi134.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-18T10:34:12.737 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-18T10:34:12.749 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2024-09-18T10:34:13.041 INFO:teuthology.orchestra.run.smithi134.stderr:instructed osd(s) 6 to deep-scrub 2024-09-18T10:34:13.051 INFO:tasks.ceph:Scrubbing osd.7 2024-09-18T10:34:13.051 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2024-09-18T10:34:13.192 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-18T10:34:13.192 INFO:teuthology.orchestra.run.smithi134.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-18T10:34:13.192 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-18T10:34:13.204 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2024-09-18T10:34:13.503 INFO:teuthology.orchestra.run.smithi134.stderr:instructed osd(s) 7 to deep-scrub 2024-09-18T10:34:13.513 INFO:tasks.ceph:Scrubbing osd.8 2024-09-18T10:34:13.513 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.8 config set osd_debug_deep_scrub_sleep 0 2024-09-18T10:34:13.651 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-18T10:34:13.651 INFO:teuthology.orchestra.run.smithi134.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-18T10:34:13.652 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-18T10:34:13.669 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 8 2024-09-18T10:34:13.965 INFO:teuthology.orchestra.run.smithi134.stderr:instructed osd(s) 8 to deep-scrub 2024-09-18T10:34:13.975 INFO:tasks.ceph:Scrubbing osd.9 2024-09-18T10:34:13.975 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.9 config set osd_debug_deep_scrub_sleep 0 2024-09-18T10:34:14.122 INFO:teuthology.orchestra.run.smithi134.stdout:{ 2024-09-18T10:34:14.122 INFO:teuthology.orchestra.run.smithi134.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-18T10:34:14.122 INFO:teuthology.orchestra.run.smithi134.stdout:} 2024-09-18T10:34:14.134 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 9 2024-09-18T10:34:14.433 INFO:teuthology.orchestra.run.smithi134.stderr:instructed osd(s) 9 to deep-scrub 2024-09-18T10:34:14.444 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-18T10:34:14.743 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-18T10:34:14.745 INFO:teuthology.orchestra.run.smithi134.stderr:dumped all 2024-09-18T10:34:14.756 INFO:teuthology.orchestra.run.smithi134.stdout:{"pg_ready":true,"pg_map":{"version":110,"stamp":"2024-09-18T10:34:13.433980+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":10,"num_per_pool_osds":10,"num_per_pool_omap_osds":8,"kb":1048576000,"kb_used":2980332,"kb_used_data":6728,"kb_used_omap":16,"kb_used_meta":2973423,"kb_avail":1045595668,"statfs":{"total":1073741824000,"available":1070689964032,"internally_reserved":0,"allocated":6889472,"data_stored":3327748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":17040,"internal_metadata":3044785520},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-2,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.303834"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":47,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-18T10:34:02.428277+0000","last_change":"2024-09-18T10:33:52.036028+0000","last_active":"2024-09-18T10:34:02.428277+0000","last_peered":"2024-09-18T10:34:02.428277+0000","last_clean":"2024-09-18T10:34:02.428277+0000","last_became_active":"2024-09-18T10:33:52.035591+0000","last_became_peered":"2024-09-18T10:33:52.035591+0000","last_unstale":"2024-09-18T10:34:02.428277+0000","last_undegraded":"2024-09-18T10:34:02.428277+0000","last_fullsized":"2024-09-18T10:34:02.428277+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T10:32:19.443095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:32:19.443095+0000","last_clean_scrub_stamp":"2024-09-18T10:32:19.443095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T18:06:23.838644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00038275899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":63,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-18T10:34:02.778343+0000","last_change":"2024-09-18T10:33:52.042125+0000","last_active":"2024-09-18T10:34:02.778343+0000","last_peered":"2024-09-18T10:34:02.778343+0000","last_clean":"2024-09-18T10:34:02.778343+0000","last_became_active":"2024-09-18T10:33:52.041895+0000","last_became_peered":"2024-09-18T10:33:52.041895+0000","last_unstale":"2024-09-18T10:34:02.778343+0000","last_undegraded":"2024-09-18T10:34:02.778343+0000","last_fullsized":"2024-09-18T10:34:02.778343+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T10:32:19.443095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:32:19.443095+0000","last_clean_scrub_stamp":"2024-09-18T10:32:19.443095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T15:18:02.048451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034975400000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":96,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-18T10:34:03.594986+0000","last_change":"2024-09-18T10:34:03.594986+0000","last_active":"2024-09-18T10:34:03.594986+0000","last_peered":"2024-09-18T10:34:03.594986+0000","last_clean":"2024-09-18T10:34:03.594986+0000","last_became_active":"2024-09-18T10:34:03.594428+0000","last_became_peered":"2024-09-18T10:34:03.594428+0000","last_unstale":"2024-09-18T10:34:03.594986+0000","last_undegraded":"2024-09-18T10:34:03.594986+0000","last_fullsized":"2024-09-18T10:34:03.594986+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T10:32:19.443095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:32:19.443095+0000","last_clean_scrub_stamp":"2024-09-18T10:32:19.443095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T18:25:37.752825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00048480000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0],"acting":[9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":62,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-18T10:34:03.532824+0000","last_change":"2024-09-18T10:34:03.531946+0000","last_active":"2024-09-18T10:34:03.532824+0000","last_peered":"2024-09-18T10:34:03.532824+0000","last_clean":"2024-09-18T10:34:03.532824+0000","last_became_active":"2024-09-18T10:34:03.531651+0000","last_became_peered":"2024-09-18T10:34:03.531651+0000","last_unstale":"2024-09-18T10:34:03.532824+0000","last_undegraded":"2024-09-18T10:34:03.532824+0000","last_fullsized":"2024-09-18T10:34:03.532824+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T10:32:19.443095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:32:19.443095+0000","last_clean_scrub_stamp":"2024-09-18T10:32:19.443095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.3","version":"18'1","reported_seq":82,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-18T10:34:03.137967+0000","last_change":"2024-09-18T10:34:03.137967+0000","last_active":"2024-09-18T10:34:03.137967+0000","last_peered":"2024-09-18T10:34:03.137967+0000","last_clean":"2024-09-18T10:34:03.137967+0000","last_became_active":"2024-09-18T10:34:03.137466+0000","last_became_peered":"2024-09-18T10:34:03.137466+0000","last_unstale":"2024-09-18T10:34:03.137967+0000","last_undegraded":"2024-09-18T10:34:03.137967+0000","last_fullsized":"2024-09-18T10:34:03.137967+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T10:32:19.443095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:32:19.443095+0000","last_clean_scrub_stamp":"2024-09-18T10:32:19.443095+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T19:22:54.145211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.2","version":"20'2","reported_seq":85,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-18T10:34:03.136912+0000","last_change":"2024-09-18T10:34:03.136912+0000","last_active":"2024-09-18T10:34:03.136912+0000","last_peered":"2024-09-18T10:34:03.136912+0000","last_clean":"2024-09-18T10:34:03.136912+0000","last_became_active":"2024-09-18T10:34:03.136418+0000","last_became_peered":"2024-09-18T10:34:03.136418+0000","last_unstale":"2024-09-18T10:34:03.136912+0000","last_undegraded":"2024-09-18T10:34:03.136912+0000","last_fullsized":"2024-09-18T10:34:03.136912+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T10:32:19.443095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:32:19.443095+0000","last_clean_scrub_stamp":"2024-09-18T10:32:19.443095+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T21:39:17.714747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0019475040000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":44,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-09-18T10:34:02.562266+0000","last_change":"2024-09-18T10:33:52.038532+0000","last_active":"2024-09-18T10:34:02.562266+0000","last_peered":"2024-09-18T10:34:02.562266+0000","last_clean":"2024-09-18T10:34:02.562266+0000","last_became_active":"2024-09-18T10:33:52.038239+0000","last_became_peered":"2024-09-18T10:33:52.038239+0000","last_unstale":"2024-09-18T10:34:02.562266+0000","last_undegraded":"2024-09-18T10:34:02.562266+0000","last_fullsized":"2024-09-18T10:34:02.562266+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T10:32:19.443095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:32:19.443095+0000","last_clean_scrub_stamp":"2024-09-18T10:32:19.443095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T18:34:12.717302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062722600000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":63,"reported_epoch":57,"state":"active+clean","last_fresh":"2024-09-18T10:34:00.270430+0000","last_change":"2024-09-18T10:33:52.037375+0000","last_active":"2024-09-18T10:34:00.270430+0000","last_peered":"2024-09-18T10:34:00.270430+0000","last_clean":"2024-09-18T10:34:00.270430+0000","last_became_active":"2024-09-18T10:33:52.037028+0000","last_became_peered":"2024-09-18T10:33:52.037028+0000","last_unstale":"2024-09-18T10:34:00.270430+0000","last_undegraded":"2024-09-18T10:34:00.270430+0000","last_fullsized":"2024-09-18T10:34:00.270430+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T10:32:19.443095+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:32:19.443095+0000","last_clean_scrub_stamp":"2024-09-18T10:32:19.443095+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T19:41:50.364436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9],"acting":[5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":9}],"osd_stats":[{"osd":9,"up_from":41,"seq":176093659161,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":298036,"kb_used_data":672,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":104559564,"statfs":{"total":107374182400,"available":107068993536,"internally_reserved":0,"allocated":688128,"data_stored":332771,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,6,7,8],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 10:33:18 2024","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":1.1930000000000001,"5min":1.1930000000000001,"15min":1.1930000000000001},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.3029999999999999,"5min":1.3029999999999999,"15min":1.3029999999999999},"last":1.216}]},{"osd":1,"last update":"Wed Sep 18 10:33:18 2024","interfaces":[{"interface":"back","average":{"1min":0.873,"5min":0.873,"15min":0.873},"min":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.3919999999999999,"5min":1.3919999999999999,"15min":1.3919999999999999},"last":0.99399999999999999},{"interface":"front","average":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":2.4359999999999999,"5min":2.4359999999999999,"15min":2.4359999999999999},"last":1.032}]},{"osd":2,"last update":"Wed Sep 18 10:33:18 2024","interfaces":[{"interface":"back","average":{"1min":1.03,"5min":1.03,"15min":1.03},"min":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":1.526,"5min":1.526,"15min":1.526},"last":1.081},{"interface":"front","average":{"1min":1.044,"5min":1.044,"15min":1.044},"min":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"max":{"1min":1.591,"5min":1.591,"15min":1.591},"last":1.141}]},{"osd":3,"last update":"Wed Sep 18 10:33:18 2024","interfaces":[{"interface":"back","average":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.542,"5min":1.542,"15min":1.542},"last":1.351},{"interface":"front","average":{"1min":1.274,"5min":1.274,"15min":1.274},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.8440000000000001,"5min":1.8440000000000001,"15min":1.8440000000000001},"last":1.181}]},{"osd":4,"last update":"Wed Sep 18 10:33:18 2024","interfaces":[{"interface":"back","average":{"1min":1.2430000000000001,"5min":1.2430000000000001,"15min":1.2430000000000001},"min":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"max":{"1min":1.8180000000000001,"5min":1.8180000000000001,"15min":1.8180000000000001},"last":1.2689999999999999},{"interface":"front","average":{"1min":1.276,"5min":1.276,"15min":1.276},"min":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"max":{"1min":1.681,"5min":1.681,"15min":1.681},"last":1.629}]},{"osd":5,"last update":"Wed Sep 18 10:33:18 2024","interfaces":[{"interface":"back","average":{"1min":1.3140000000000001,"5min":1.3140000000000001,"15min":1.3140000000000001},"min":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"max":{"1min":1.8600000000000001,"5min":1.8600000000000001,"15min":1.8600000000000001},"last":1.7450000000000001},{"interface":"front","average":{"1min":1.3009999999999999,"5min":1.3009999999999999,"15min":1.3009999999999999},"min":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"max":{"1min":1.7090000000000001,"5min":1.7090000000000001,"15min":1.7090000000000001},"last":1.5569999999999999}]},{"osd":6,"last update":"Wed Sep 18 10:33:18 2024","interfaces":[{"interface":"back","average":{"1min":1.3029999999999999,"5min":1.3029999999999999,"15min":1.3029999999999999},"min":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":2.778,"5min":2.778,"15min":2.778},"last":1.6639999999999999},{"interface":"front","average":{"1min":1.345,"5min":1.345,"15min":1.345},"min":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":1.921,"5min":1.921,"15min":1.921},"last":1.329}]},{"osd":7,"last update":"Wed Sep 18 10:33:18 2024","interfaces":[{"interface":"back","average":{"1min":1.3340000000000001,"5min":1.3340000000000001,"15min":1.3340000000000001},"min":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"max":{"1min":1.6830000000000001,"5min":1.6830000000000001,"15min":1.6830000000000001},"last":1.393},{"interface":"front","average":{"1min":1.417,"5min":1.417,"15min":1.417},"min":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"max":{"1min":3.0390000000000001,"5min":3.0390000000000001,"15min":3.0390000000000001},"last":1.7}]},{"osd":8,"last update":"Wed Sep 18 10:33:18 2024","interfaces":[{"interface":"back","average":{"1min":1.4330000000000001,"5min":1.4330000000000001,"15min":1.4330000000000001},"min":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"max":{"1min":2.8500000000000001,"5min":2.8500000000000001,"15min":2.8500000000000001},"last":1.7290000000000001},{"interface":"front","average":{"1min":1.4079999999999999,"5min":1.4079999999999999,"15min":1.4079999999999999},"min":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"max":{"1min":2.0070000000000001,"5min":2.0070000000000001,"15min":2.0070000000000001},"last":1.458}]}]},{"osd":8,"up_from":48,"seq":206158430234,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":104857600,"kb_used":298032,"kb_used_data":672,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":104559568,"statfs":{"total":107374182400,"available":107068997632,"internally_reserved":0,"allocated":688128,"data_stored":332771,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,6,7,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 10:33:16 2024","interfaces":[{"interface":"back","average":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":1.26,"5min":1.26,"15min":1.26},"last":0.75900000000000001},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":1.238,"5min":1.238,"15min":1.238},"last":1.296}]},{"osd":1,"last update":"Wed Sep 18 10:33:16 2024","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.751,"15min":0.751},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":1.129,"5min":1.129,"15min":1.129},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"min":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.2390000000000001,"5min":1.2390000000000001,"15min":1.2390000000000001},"last":1.0489999999999999}]},{"osd":2,"last update":"Wed Sep 18 10:33:16 2024","interfaces":[{"interface":"back","average":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":1.6439999999999999,"5min":1.6439999999999999,"15min":1.6439999999999999},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":1.2529999999999999,"5min":1.2529999999999999,"15min":1.2529999999999999},"last":0.79500000000000004}]},{"osd":3,"last update":"Wed Sep 18 10:33:16 2024","interfaces":[{"interface":"back","average":{"1min":1.109,"5min":1.109,"15min":1.109},"min":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"max":{"1min":1.538,"5min":1.538,"15min":1.538},"last":1.1539999999999999},{"interface":"front","average":{"1min":1.0389999999999999,"5min":1.0389999999999999,"15min":1.0389999999999999},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.4810000000000001,"5min":1.4810000000000001,"15min":1.4810000000000001},"last":1.2}]},{"osd":4,"last update":"Wed Sep 18 10:33:16 2024","interfaces":[{"interface":"back","average":{"1min":1.0620000000000001,"5min":1.0620000000000001,"15min":1.0620000000000001},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.8169999999999999,"5min":1.8169999999999999,"15min":1.8169999999999999},"last":1.1299999999999999},{"interface":"front","average":{"1min":1.2549999999999999,"5min":1.2549999999999999,"15min":1.2549999999999999},"min":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"max":{"1min":2.1480000000000001,"5min":2.1480000000000001,"15min":2.1480000000000001},"last":1.4039999999999999}]},{"osd":5,"last update":"Wed Sep 18 10:33:16 2024","interfaces":[{"interface":"back","average":{"1min":1.216,"5min":1.216,"15min":1.216},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":2.3109999999999999,"5min":2.3109999999999999,"15min":2.3109999999999999},"last":1.4670000000000001},{"interface":"front","average":{"1min":1.161,"5min":1.161,"15min":1.161},"min":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":2.0169999999999999,"5min":2.0169999999999999,"15min":2.0169999999999999},"last":1.3819999999999999}]},{"osd":6,"last update":"Wed Sep 18 10:33:16 2024","interfaces":[{"interface":"back","average":{"1min":1.2889999999999999,"5min":1.2889999999999999,"15min":1.2889999999999999},"min":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"max":{"1min":2.1899999999999999,"5min":2.1899999999999999,"15min":2.1899999999999999},"last":1.222},{"interface":"front","average":{"1min":1.1799999999999999,"5min":1.1799999999999999,"15min":1.1799999999999999},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":2.069,"5min":2.069,"15min":2.069},"last":1.3520000000000001}]},{"osd":7,"last update":"Wed Sep 18 10:33:16 2024","interfaces":[{"interface":"back","average":{"1min":1.2589999999999999,"5min":1.2589999999999999,"15min":1.2589999999999999},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":2.2130000000000001,"5min":2.2130000000000001,"15min":2.2130000000000001},"last":1.54},{"interface":"front","average":{"1min":1.4039999999999999,"5min":1.4039999999999999,"15min":1.4039999999999999},"min":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"max":{"1min":2.2450000000000001,"5min":2.2450000000000001,"15min":2.2450000000000001},"last":1.4870000000000001}]},{"osd":9,"last update":"Wed Sep 18 10:33:16 2024","interfaces":[{"interface":"back","average":{"1min":1.304,"5min":1.304,"15min":1.304},"min":{"1min":0.63,"5min":0.63,"15min":0.63},"max":{"1min":2.1800000000000002,"5min":2.1800000000000002,"15min":2.1800000000000002},"last":1.512},{"interface":"front","average":{"1min":1.3440000000000001,"5min":1.3440000000000001,"15min":1.3440000000000001},"min":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"max":{"1min":2.3969999999999998,"5min":2.3969999999999998,"15min":2.3969999999999998},"last":1.593}]}]},{"osd":7,"up_from":48,"seq":206158430233,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":298032,"kb_used_data":672,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":104559568,"statfs":{"total":107374182400,"available":107068997632,"internally_reserved":0,"allocated":688128,"data_stored":332771,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,6,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 10:33:17 2024","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.202,"5min":1.202,"15min":1.202},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.75,"5min":0.75,"15min":0.75},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":1.486,"5min":1.486,"15min":1.486},"last":0.91500000000000004}]},{"osd":1,"last update":"Wed Sep 18 10:33:17 2024","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.3009999999999999,"5min":1.3009999999999999,"15min":1.3009999999999999},"last":0.78100000000000003},{"interface":"front","average":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.8480000000000001,"5min":1.8480000000000001,"15min":1.8480000000000001},"last":0.71799999999999997}]},{"osd":2,"last update":"Wed Sep 18 10:33:17 2024","interfaces":[{"interface":"back","average":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":1.5980000000000001,"5min":1.5980000000000001,"15min":1.5980000000000001},"last":0.97999999999999998},{"interface":"front","average":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.3440000000000001,"5min":1.3440000000000001,"15min":1.3440000000000001},"last":1.0900000000000001}]},{"osd":3,"last update":"Wed Sep 18 10:33:17 2024","interfaces":[{"interface":"back","average":{"1min":1.1000000000000001,"5min":1.1000000000000001,"15min":1.1000000000000001},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.9319999999999999,"5min":1.9319999999999999,"15min":1.9319999999999999},"last":1.0600000000000001},{"interface":"front","average":{"1min":1.0720000000000001,"5min":1.0720000000000001,"15min":1.0720000000000001},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.8879999999999999,"5min":1.8879999999999999,"15min":1.8879999999999999},"last":1.167}]},{"osd":4,"last update":"Wed Sep 18 10:33:17 2024","interfaces":[{"interface":"back","average":{"1min":1.097,"5min":1.097,"15min":1.097},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":2.1379999999999999,"5min":2.1379999999999999,"15min":2.1379999999999999},"last":1.6080000000000001},{"interface":"front","average":{"1min":1.1060000000000001,"5min":1.1060000000000001,"15min":1.1060000000000001},"min":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":2.0110000000000001,"5min":2.0110000000000001,"15min":2.0110000000000001},"last":1.1299999999999999}]},{"osd":5,"last update":"Wed Sep 18 10:33:17 2024","interfaces":[{"interface":"back","average":{"1min":1.194,"5min":1.194,"15min":1.194},"min":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"max":{"1min":2.4889999999999999,"5min":2.4889999999999999,"15min":2.4889999999999999},"last":1.2430000000000001},{"interface":"front","average":{"1min":1.2230000000000001,"5min":1.2230000000000001,"15min":1.2230000000000001},"min":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"max":{"1min":2.4289999999999998,"5min":2.4289999999999998,"15min":2.4289999999999998},"last":1.7030000000000001}]},{"osd":6,"last update":"Wed Sep 18 10:33:17 2024","interfaces":[{"interface":"back","average":{"1min":1.2769999999999999,"5min":1.2769999999999999,"15min":1.2769999999999999},"min":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"max":{"1min":2.0950000000000002,"5min":2.0950000000000002,"15min":2.0950000000000002},"last":1.196},{"interface":"front","average":{"1min":1.218,"5min":1.218,"15min":1.218},"min":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":1.9450000000000001,"5min":1.9450000000000001,"15min":1.9450000000000001},"last":1.3320000000000001}]},{"osd":8,"last update":"Wed Sep 18 10:33:17 2024","interfaces":[{"interface":"back","average":{"1min":1.2969999999999999,"5min":1.2969999999999999,"15min":1.2969999999999999},"min":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"max":{"1min":2.6389999999999998,"5min":2.6389999999999998,"15min":2.6389999999999998},"last":1.4450000000000001},{"interface":"front","average":{"1min":1.3600000000000001,"5min":1.3600000000000001,"15min":1.3600000000000001},"min":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"max":{"1min":2.1520000000000001,"5min":2.1520000000000001,"15min":2.1520000000000001},"last":1.5700000000000001}]},{"osd":9,"last update":"Wed Sep 18 10:33:17 2024","interfaces":[{"interface":"back","average":{"1min":1.2470000000000001,"5min":1.2470000000000001,"15min":1.2470000000000001},"min":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":2.0680000000000001,"5min":2.0680000000000001,"15min":2.0680000000000001},"last":1.6659999999999999},{"interface":"front","average":{"1min":1.282,"5min":1.282,"15min":1.282},"min":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"max":{"1min":1.8460000000000001,"5min":1.8460000000000001,"15min":1.8460000000000001},"last":1.5209999999999999}]}]},{"osd":6,"up_from":48,"seq":206158430233,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":298032,"kb_used_data":672,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":104559568,"statfs":{"total":107374182400,"available":107068997632,"internally_reserved":0,"allocated":688128,"data_stored":332771,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,7,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 10:33:17 2024","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":1.073,"5min":1.073,"15min":1.073},"last":1.0429999999999999},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":1.3720000000000001,"5min":1.3720000000000001,"15min":1.3720000000000001},"last":0.69099999999999995}]},{"osd":1,"last update":"Wed Sep 18 10:33:17 2024","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.751,"15min":0.751},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":1.6599999999999999,"5min":1.6599999999999999,"15min":1.6599999999999999},"last":0.80500000000000005},{"interface":"front","average":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":2.0459999999999998,"5min":2.0459999999999998,"15min":2.0459999999999998},"last":1.2549999999999999}]},{"osd":2,"last update":"Wed Sep 18 10:33:17 2024","interfaces":[{"interface":"back","average":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":1.1399999999999999,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":0.71799999999999997},{"interface":"front","average":{"1min":1.052,"5min":1.052,"15min":1.052},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":1.907,"5min":1.907,"15min":1.907},"last":1.5189999999999999}]},{"osd":3,"last update":"Wed Sep 18 10:33:17 2024","interfaces":[{"interface":"back","average":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.7490000000000001,"5min":1.7490000000000001,"15min":1.7490000000000001},"last":1.3660000000000001},{"interface":"front","average":{"1min":1.1830000000000001,"5min":1.1830000000000001,"15min":1.1830000000000001},"min":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"max":{"1min":1.8680000000000001,"5min":1.8680000000000001,"15min":1.8680000000000001},"last":0.96099999999999997}]},{"osd":4,"last update":"Wed Sep 18 10:33:17 2024","interfaces":[{"interface":"back","average":{"1min":1.2070000000000001,"5min":1.2070000000000001,"15min":1.2070000000000001},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.976,"5min":1.976,"15min":1.976},"last":1.288},{"interface":"front","average":{"1min":1.127,"5min":1.127,"15min":1.127},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":2.544,"5min":2.544,"15min":2.544},"last":0.999}]},{"osd":5,"last update":"Wed Sep 18 10:33:17 2024","interfaces":[{"interface":"back","average":{"1min":1.2170000000000001,"5min":1.2170000000000001,"15min":1.2170000000000001},"min":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"max":{"1min":2.1539999999999999,"5min":2.1539999999999999,"15min":2.1539999999999999},"last":1.175},{"interface":"front","average":{"1min":1.296,"5min":1.296,"15min":1.296},"min":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":2.2469999999999999,"5min":2.2469999999999999,"15min":2.2469999999999999},"last":1.327}]},{"osd":7,"last update":"Wed Sep 18 10:33:17 2024","interfaces":[{"interface":"back","average":{"1min":1.246,"5min":1.246,"15min":1.246},"min":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":2.6440000000000001,"5min":2.6440000000000001,"15min":2.6440000000000001},"last":1.673},{"interface":"front","average":{"1min":1.7609999999999999,"5min":1.7609999999999999,"15min":1.7609999999999999},"min":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"max":{"1min":10.385,"5min":10.385,"15min":10.385},"last":1.3999999999999999}]},{"osd":8,"last update":"Wed Sep 18 10:33:17 2024","interfaces":[{"interface":"back","average":{"1min":1.3640000000000001,"5min":1.3640000000000001,"15min":1.3640000000000001},"min":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"max":{"1min":2.444,"5min":2.444,"15min":2.444},"last":1.218},{"interface":"front","average":{"1min":1.274,"5min":1.274,"15min":1.274},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":2.714,"5min":2.714,"15min":2.714},"last":1.7270000000000001}]},{"osd":9,"last update":"Wed Sep 18 10:33:17 2024","interfaces":[{"interface":"back","average":{"1min":1.2789999999999999,"5min":1.2789999999999999,"15min":1.2789999999999999},"min":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"max":{"1min":2.2879999999999998,"5min":2.2879999999999998,"15min":2.2879999999999998},"last":1.476},{"interface":"front","average":{"1min":1.3240000000000001,"5min":1.3240000000000001,"15min":1.3240000000000001},"min":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":2.5960000000000001,"5min":2.5960000000000001,"15min":2.5960000000000001},"last":1.4279999999999999}]}]},{"osd":5,"up_from":48,"seq":206158430235,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":298028,"kb_used_data":672,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":104559572,"statfs":{"total":107374182400,"available":107069001728,"internally_reserved":0,"allocated":688128,"data_stored":332771,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,6,7,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 10:33:18 2024","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.111,"5min":1.111,"15min":1.111},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"min":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.5249999999999999,"5min":1.5249999999999999,"15min":1.5249999999999999},"last":1.1279999999999999}]},{"osd":1,"last update":"Wed Sep 18 10:33:18 2024","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.173,"5min":1.173,"15min":1.173},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.8280000000000001,"5min":1.8280000000000001,"15min":1.8280000000000001},"last":0.85299999999999998}]},{"osd":2,"last update":"Wed Sep 18 10:33:18 2024","interfaces":[{"interface":"back","average":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"min":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":1.4159999999999999,"5min":1.4159999999999999,"15min":1.4159999999999999},"last":1.268},{"interface":"front","average":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"min":{"1min":0.627,"5min":0.627,"15min":0.627},"max":{"1min":2.3849999999999998,"5min":2.3849999999999998,"15min":2.3849999999999998},"last":1.103}]},{"osd":3,"last update":"Wed Sep 18 10:33:18 2024","interfaces":[{"interface":"back","average":{"1min":1.079,"5min":1.079,"15min":1.079},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.923,"5min":1.923,"15min":1.923},"last":1.3180000000000001},{"interface":"front","average":{"1min":1.268,"5min":1.268,"15min":1.268},"min":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"max":{"1min":2.073,"5min":2.073,"15min":2.073},"last":0.95299999999999996}]},{"osd":4,"last update":"Wed Sep 18 10:33:18 2024","interfaces":[{"interface":"back","average":{"1min":1.123,"5min":1.123,"15min":1.123},"min":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"max":{"1min":2.141,"5min":2.141,"15min":2.141},"last":1.0720000000000001},{"interface":"front","average":{"1min":1.2490000000000001,"5min":1.2490000000000001,"15min":1.2490000000000001},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":2.3300000000000001,"5min":2.3300000000000001,"15min":2.3300000000000001},"last":1.179}]},{"osd":6,"last update":"Wed Sep 18 10:33:18 2024","interfaces":[{"interface":"back","average":{"1min":1.2769999999999999,"5min":1.2769999999999999,"15min":1.2769999999999999},"min":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"max":{"1min":1.806,"5min":1.806,"15min":1.806},"last":1.0529999999999999},{"interface":"front","average":{"1min":1.274,"5min":1.274,"15min":1.274},"min":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"max":{"1min":2.6120000000000001,"5min":2.6120000000000001,"15min":2.6120000000000001},"last":1.2330000000000001}]},{"osd":7,"last update":"Wed Sep 18 10:33:18 2024","interfaces":[{"interface":"back","average":{"1min":1.3380000000000001,"5min":1.3380000000000001,"15min":1.3380000000000001},"min":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"max":{"1min":2.661,"5min":2.661,"15min":2.661},"last":1.01},{"interface":"front","average":{"1min":1.3819999999999999,"5min":1.3819999999999999,"15min":1.3819999999999999},"min":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"max":{"1min":2.548,"5min":2.548,"15min":2.548},"last":1.1539999999999999}]},{"osd":8,"last update":"Wed Sep 18 10:33:18 2024","interfaces":[{"interface":"back","average":{"1min":1.359,"5min":1.359,"15min":1.359},"min":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"max":{"1min":2.7080000000000002,"5min":2.7080000000000002,"15min":2.7080000000000002},"last":1.5629999999999999},{"interface":"front","average":{"1min":1.327,"5min":1.327,"15min":1.327},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":2.4399999999999999,"5min":2.4399999999999999,"15min":2.4399999999999999},"last":1.7110000000000001}]},{"osd":9,"last update":"Wed Sep 18 10:33:18 2024","interfaces":[{"interface":"back","average":{"1min":1.3859999999999999,"5min":1.3859999999999999,"15min":1.3859999999999999},"min":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"max":{"1min":2.2130000000000001,"5min":2.2130000000000001,"15min":2.2130000000000001},"last":1.6399999999999999},{"interface":"front","average":{"1min":1.4019999999999999,"5min":1.4019999999999999,"15min":1.4019999999999999},"min":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"max":{"1min":2.782,"5min":2.782,"15min":2.782},"last":1.6830000000000001}]}]},{"osd":4,"up_from":48,"seq":206158430233,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":298036,"kb_used_data":676,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":104559564,"statfs":{"total":107374182400,"available":107068993536,"internally_reserved":0,"allocated":692224,"data_stored":332790,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,5,6,7,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 10:33:16 2024","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":1.0960000000000001,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.745,"5min":0.745,"15min":0.745},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.171,"5min":1.171,"15min":1.171},"last":0.86399999999999999}]},{"osd":1,"last update":"Wed Sep 18 10:33:16 2024","interfaces":[{"interface":"back","average":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":1.1579999999999999,"5min":1.1579999999999999,"15min":1.1579999999999999},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.583,"5min":1.583,"15min":1.583},"last":0.94599999999999995}]},{"osd":2,"last update":"Wed Sep 18 10:33:16 2024","interfaces":[{"interface":"back","average":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":2.1139999999999999,"5min":2.1139999999999999,"15min":2.1139999999999999},"last":1.0800000000000001},{"interface":"front","average":{"1min":0.995,"5min":0.995,"15min":0.995},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.5660000000000001,"5min":1.5660000000000001,"15min":1.5660000000000001},"last":1.3300000000000001}]},{"osd":3,"last update":"Wed Sep 18 10:33:16 2024","interfaces":[{"interface":"back","average":{"1min":1.0620000000000001,"5min":1.0620000000000001,"15min":1.0620000000000001},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.6200000000000001,"5min":1.6200000000000001,"15min":1.6200000000000001},"last":0.82799999999999996},{"interface":"front","average":{"1min":1.1699999999999999,"5min":1.1699999999999999,"15min":1.1699999999999999},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.732,"5min":1.732,"15min":1.732},"last":1.284}]},{"osd":5,"last update":"Wed Sep 18 10:33:16 2024","interfaces":[{"interface":"back","average":{"1min":1.159,"5min":1.159,"15min":1.159},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":2.2509999999999999,"5min":2.2509999999999999,"15min":2.2509999999999999},"last":1.4279999999999999},{"interface":"front","average":{"1min":1.1830000000000001,"5min":1.1830000000000001,"15min":1.1830000000000001},"min":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":1.873,"5min":1.873,"15min":1.873},"last":0.90900000000000003}]},{"osd":6,"last update":"Wed Sep 18 10:33:16 2024","interfaces":[{"interface":"back","average":{"1min":1.214,"5min":1.214,"15min":1.214},"min":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"max":{"1min":2.2879999999999998,"5min":2.2879999999999998,"15min":2.2879999999999998},"last":1.3779999999999999},{"interface":"front","average":{"1min":1.25,"5min":1.25,"15min":1.25},"min":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":2.661,"5min":2.661,"15min":2.661},"last":1.1439999999999999}]},{"osd":7,"last update":"Wed Sep 18 10:33:16 2024","interfaces":[{"interface":"back","average":{"1min":1.319,"5min":1.319,"15min":1.319},"min":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"max":{"1min":1.895,"5min":1.895,"15min":1.895},"last":1.52},{"interface":"front","average":{"1min":1.2450000000000001,"5min":1.2450000000000001,"15min":1.2450000000000001},"min":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":2.0579999999999998,"5min":2.0579999999999998,"15min":2.0579999999999998},"last":1.403}]},{"osd":8,"last update":"Wed Sep 18 10:33:16 2024","interfaces":[{"interface":"back","average":{"1min":1.278,"5min":1.278,"15min":1.278},"min":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":2.173,"5min":2.173,"15min":2.173},"last":1.49},{"interface":"front","average":{"1min":1.3240000000000001,"5min":1.3240000000000001,"15min":1.3240000000000001},"min":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":1.9750000000000001,"5min":1.9750000000000001,"15min":1.9750000000000001},"last":1.4650000000000001}]},{"osd":9,"last update":"Wed Sep 18 10:33:16 2024","interfaces":[{"interface":"back","average":{"1min":1.327,"5min":1.327,"15min":1.327},"min":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":2.448,"5min":2.448,"15min":2.448},"last":1.546},{"interface":"front","average":{"1min":1.389,"5min":1.389,"15min":1.389},"min":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"max":{"1min":2.242,"5min":2.242,"15min":2.242},"last":1.573}]}]},{"osd":3,"up_from":48,"seq":206158430235,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":298032,"kb_used_data":672,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":104559568,"statfs":{"total":107374182400,"available":107068997632,"internally_reserved":0,"allocated":688128,"data_stored":332771,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,4,5,6,7,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 10:33:15 2024","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.095,"5min":1.095,"15min":1.095},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"min":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":2.121,"5min":2.121,"15min":2.121},"last":1.127}]},{"osd":1,"last update":"Wed Sep 18 10:33:15 2024","interfaces":[{"interface":"back","average":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":1.9259999999999999,"5min":1.9259999999999999,"15min":1.9259999999999999},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"min":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":1.2090000000000001,"5min":1.2090000000000001,"15min":1.2090000000000001},"last":0.68999999999999995}]},{"osd":2,"last update":"Wed Sep 18 10:33:15 2024","interfaces":[{"interface":"back","average":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.204,"5min":1.204,"15min":1.204},"last":0.874},{"interface":"front","average":{"1min":1.1870000000000001,"5min":1.1870000000000001,"15min":1.1870000000000001},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":3.2040000000000002,"5min":3.2040000000000002,"15min":3.2040000000000002},"last":0.82299999999999995}]},{"osd":4,"last update":"Wed Sep 18 10:33:15 2024","interfaces":[{"interface":"back","average":{"1min":1.071,"5min":1.071,"15min":1.071},"min":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":1.49,"5min":1.49,"15min":1.49},"last":1.0409999999999999},{"interface":"front","average":{"1min":1.266,"5min":1.266,"15min":1.266},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":2.7519999999999998,"5min":2.7519999999999998,"15min":2.7519999999999998},"last":1.3120000000000001}]},{"osd":5,"last update":"Wed Sep 18 10:33:15 2024","interfaces":[{"interface":"back","average":{"1min":1.1830000000000001,"5min":1.1830000000000001,"15min":1.1830000000000001},"min":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"max":{"1min":2.0630000000000002,"5min":2.0630000000000002,"15min":2.0630000000000002},"last":1.4339999999999999},{"interface":"front","average":{"1min":1.1259999999999999,"5min":1.1259999999999999,"15min":1.1259999999999999},"min":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"max":{"1min":1.5149999999999999,"5min":1.5149999999999999,"15min":1.5149999999999999},"last":1.093}]},{"osd":6,"last update":"Wed Sep 18 10:33:15 2024","interfaces":[{"interface":"back","average":{"1min":1.3620000000000001,"5min":1.3620000000000001,"15min":1.3620000000000001},"min":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":4.1029999999999998,"5min":4.1029999999999998,"15min":4.1029999999999998},"last":1.274},{"interface":"front","average":{"1min":1.268,"5min":1.268,"15min":1.268},"min":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"max":{"1min":2.2789999999999999,"5min":2.2789999999999999,"15min":2.2789999999999999},"last":1.353}]},{"osd":7,"last update":"Wed Sep 18 10:33:15 2024","interfaces":[{"interface":"back","average":{"1min":1.2010000000000001,"5min":1.2010000000000001,"15min":1.2010000000000001},"min":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"max":{"1min":2.1880000000000002,"5min":2.1880000000000002,"15min":2.1880000000000002},"last":1.2070000000000001},{"interface":"front","average":{"1min":1.282,"5min":1.282,"15min":1.282},"min":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"max":{"1min":3.0409999999999999,"5min":3.0409999999999999,"15min":3.0409999999999999},"last":1.079}]},{"osd":8,"last update":"Wed Sep 18 10:33:15 2024","interfaces":[{"interface":"back","average":{"1min":1.5109999999999999,"5min":1.5109999999999999,"15min":1.5109999999999999},"min":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"max":{"1min":3.5590000000000002,"5min":3.5590000000000002,"15min":3.5590000000000002},"last":1.4890000000000001},{"interface":"front","average":{"1min":1.3540000000000001,"5min":1.3540000000000001,"15min":1.3540000000000001},"min":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"max":{"1min":2.3919999999999999,"5min":2.3919999999999999,"15min":2.3919999999999999},"last":1.167}]},{"osd":9,"last update":"Wed Sep 18 10:33:15 2024","interfaces":[{"interface":"back","average":{"1min":1.379,"5min":1.379,"15min":1.379},"min":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"max":{"1min":3.1259999999999999,"5min":3.1259999999999999,"15min":3.1259999999999999},"last":1.534},{"interface":"front","average":{"1min":1.5109999999999999,"5min":1.5109999999999999,"15min":1.5109999999999999},"min":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"max":{"1min":3.6339999999999999,"5min":3.6339999999999999,"15min":3.6339999999999999},"last":1.3959999999999999}]}]},{"osd":2,"up_from":48,"seq":206158430233,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":104857600,"kb_used":298028,"kb_used_data":672,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":104559572,"statfs":{"total":107374182400,"available":107069001728,"internally_reserved":0,"allocated":688128,"data_stored":332771,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,3,4,5,6,7,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 10:33:18 2024","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1.109,"5min":1.109,"15min":1.109},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":1.5029999999999999,"5min":1.5029999999999999,"15min":1.5029999999999999},"last":1.1160000000000001}]},{"osd":1,"last update":"Wed Sep 18 10:33:18 2024","interfaces":[{"interface":"back","average":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"min":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.651,"5min":1.651,"15min":1.651},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":1.1559999999999999,"5min":1.1559999999999999,"15min":1.1559999999999999},"last":0.79700000000000004}]},{"osd":3,"last update":"Wed Sep 18 10:33:18 2024","interfaces":[{"interface":"back","average":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":1.2869999999999999,"5min":1.2869999999999999,"15min":1.2869999999999999},"last":1.3580000000000001},{"interface":"front","average":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.7390000000000001,"5min":1.7390000000000001,"15min":1.7390000000000001},"last":1.1639999999999999}]},{"osd":4,"last update":"Wed Sep 18 10:33:18 2024","interfaces":[{"interface":"back","average":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"min":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":1.333,"5min":1.333,"15min":1.333},"last":1.2090000000000001},{"interface":"front","average":{"1min":1.151,"5min":1.151,"15min":1.151},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.7869999999999999,"5min":1.7869999999999999,"15min":1.7869999999999999},"last":0.755}]},{"osd":5,"last update":"Wed Sep 18 10:33:18 2024","interfaces":[{"interface":"back","average":{"1min":1.1100000000000001,"5min":1.1100000000000001,"15min":1.1100000000000001},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.597,"5min":1.597,"15min":1.597},"last":0.93700000000000006},{"interface":"front","average":{"1min":1.246,"5min":1.246,"15min":1.246},"min":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"max":{"1min":1.873,"5min":1.873,"15min":1.873},"last":1.448}]},{"osd":6,"last update":"Wed Sep 18 10:33:18 2024","interfaces":[{"interface":"back","average":{"1min":1.0649999999999999,"5min":1.0649999999999999,"15min":1.0649999999999999},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":1.696,"5min":1.696,"15min":1.696},"last":1.48},{"interface":"front","average":{"1min":1.1479999999999999,"5min":1.1479999999999999,"15min":1.1479999999999999},"min":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"max":{"1min":1.5289999999999999,"5min":1.5289999999999999,"15min":1.5289999999999999},"last":1.327}]},{"osd":7,"last update":"Wed Sep 18 10:33:18 2024","interfaces":[{"interface":"back","average":{"1min":1.268,"5min":1.268,"15min":1.268},"min":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":1.9350000000000001,"5min":1.9350000000000001,"15min":1.9350000000000001},"last":1.282},{"interface":"front","average":{"1min":1.244,"5min":1.244,"15min":1.244},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":2.6419999999999999,"5min":2.6419999999999999,"15min":2.6419999999999999},"last":1.0169999999999999}]},{"osd":8,"last update":"Wed Sep 18 10:33:18 2024","interfaces":[{"interface":"back","average":{"1min":1.306,"5min":1.306,"15min":1.306},"min":{"1min":1,"5min":1,"15min":1},"max":{"1min":1.613,"5min":1.613,"15min":1.613},"last":1.24},{"interface":"front","average":{"1min":1.278,"5min":1.278,"15min":1.278},"min":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":1.9830000000000001,"5min":1.9830000000000001,"15min":1.9830000000000001},"last":1.4079999999999999}]},{"osd":9,"last update":"Wed Sep 18 10:33:18 2024","interfaces":[{"interface":"back","average":{"1min":1.2210000000000001,"5min":1.2210000000000001,"15min":1.2210000000000001},"min":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"max":{"1min":2.4220000000000002,"5min":2.4220000000000002,"15min":2.4220000000000002},"last":1.387},{"interface":"front","average":{"1min":1.337,"5min":1.337,"15min":1.337},"min":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"max":{"1min":2.1349999999999998,"5min":2.1349999999999998,"15min":2.1349999999999998},"last":1.512}]}]},{"osd":1,"up_from":59,"seq":253403070490,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":298040,"kb_used_data":676,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":104559560,"statfs":{"total":107374182400,"available":107068989440,"internally_reserved":0,"allocated":692224,"data_stored":332790,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,2,3,4,5,6,7,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 10:33:17 2024","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":1.0980000000000001,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.92000000000000004},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"min":{"1min":0.248,"5min":0.248,"15min":0.248},"max":{"1min":1.3,"5min":1.3,"15min":1.3},"last":0.69899999999999995}]},{"osd":2,"last update":"Wed Sep 18 10:33:17 2024","interfaces":[{"interface":"back","average":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.206,"5min":1.206,"15min":1.206},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":1.54,"5min":1.54,"15min":1.54},"last":0.64100000000000001}]},{"osd":3,"last update":"Wed Sep 18 10:33:17 2024","interfaces":[{"interface":"back","average":{"1min":1.129,"5min":1.129,"15min":1.129},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":5.7569999999999997,"5min":5.7569999999999997,"15min":5.7569999999999997},"last":0.95399999999999996},{"interface":"front","average":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":2.9830000000000001,"5min":2.9830000000000001,"15min":2.9830000000000001},"last":0.74099999999999999}]},{"osd":4,"last update":"Wed Sep 18 10:33:17 2024","interfaces":[{"interface":"back","average":{"1min":1.1000000000000001,"5min":1.1000000000000001,"15min":1.1000000000000001},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":3.8700000000000001,"5min":3.8700000000000001,"15min":3.8700000000000001},"last":0.77300000000000002},{"interface":"front","average":{"1min":1.1359999999999999,"5min":1.1359999999999999,"15min":1.1359999999999999},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":2.6520000000000001,"5min":2.6520000000000001,"15min":2.6520000000000001},"last":1.002}]},{"osd":5,"last update":"Wed Sep 18 10:33:17 2024","interfaces":[{"interface":"back","average":{"1min":1.1160000000000001,"5min":1.1160000000000001,"15min":1.1160000000000001},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":2.0470000000000002,"5min":2.0470000000000002,"15min":2.0470000000000002},"last":1.3919999999999999},{"interface":"front","average":{"1min":1.0920000000000001,"5min":1.0920000000000001,"15min":1.0920000000000001},"min":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"max":{"1min":1.6679999999999999,"5min":1.6679999999999999,"15min":1.6679999999999999},"last":1.319}]},{"osd":6,"last update":"Wed Sep 18 10:33:17 2024","interfaces":[{"interface":"back","average":{"1min":1.3280000000000001,"5min":1.3280000000000001,"15min":1.3280000000000001},"min":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":3.1299999999999999,"5min":3.1299999999999999,"15min":3.1299999999999999},"last":1.0269999999999999},{"interface":"front","average":{"1min":1.123,"5min":1.123,"15min":1.123},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.9390000000000001,"5min":1.9390000000000001,"15min":1.9390000000000001},"last":1.264}]},{"osd":7,"last update":"Wed Sep 18 10:33:17 2024","interfaces":[{"interface":"back","average":{"1min":1.1060000000000001,"5min":1.1060000000000001,"15min":1.1060000000000001},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":2.4900000000000002,"5min":2.4900000000000002,"15min":2.4900000000000002},"last":1.3620000000000001},{"interface":"front","average":{"1min":1.3120000000000001,"5min":1.3120000000000001,"15min":1.3120000000000001},"min":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":3.2810000000000001,"5min":3.2810000000000001,"15min":3.2810000000000001},"last":1.216}]},{"osd":8,"last update":"Wed Sep 18 10:33:17 2024","interfaces":[{"interface":"back","average":{"1min":1.258,"5min":1.258,"15min":1.258},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":4.4969999999999999,"5min":4.4969999999999999,"15min":4.4969999999999999},"last":1.165},{"interface":"front","average":{"1min":1.1419999999999999,"5min":1.1419999999999999,"15min":1.1419999999999999},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":2.1629999999999998,"5min":2.1629999999999998,"15min":2.1629999999999998},"last":1.131}]},{"osd":9,"last update":"Wed Sep 18 10:33:17 2024","interfaces":[{"interface":"back","average":{"1min":1.4419999999999999,"5min":1.4419999999999999,"15min":1.4419999999999999},"min":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"max":{"1min":3.2160000000000002,"5min":3.2160000000000002,"15min":3.2160000000000002},"last":1.516},{"interface":"front","average":{"1min":1.2410000000000001,"5min":1.2410000000000001,"15min":1.2410000000000001},"min":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"max":{"1min":1.9430000000000001,"5min":1.9430000000000001,"15min":1.9430000000000001},"last":1.536}]}]},{"osd":0,"up_from":59,"seq":253403070490,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":298036,"kb_used_data":672,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":104559564,"statfs":{"total":107374182400,"available":107068993536,"internally_reserved":0,"allocated":688128,"data_stored":332771,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[1,2,3,4,5,6,7,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Sep 18 10:33:15 2024","interfaces":[{"interface":"back","average":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":1.5880000000000001,"5min":1.5880000000000001,"15min":1.5880000000000001},"last":0.77900000000000003},{"interface":"front","average":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.917,"5min":1.917,"15min":1.917},"last":0.60199999999999998}]},{"osd":2,"last update":"Wed Sep 18 10:33:15 2024","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.746,"15min":0.746},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.454,"5min":1.454,"15min":1.454},"last":0.91800000000000004},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"min":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.222,"5min":1.222,"15min":1.222},"last":0.95899999999999996}]},{"osd":3,"last update":"Wed Sep 18 10:33:15 2024","interfaces":[{"interface":"back","average":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":1.5509999999999999,"5min":1.5509999999999999,"15min":1.5509999999999999},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.6339999999999999,"5min":1.6339999999999999,"15min":1.6339999999999999},"last":0.52100000000000002}]},{"osd":4,"last update":"Wed Sep 18 10:33:15 2024","interfaces":[{"interface":"back","average":{"1min":1,"5min":1,"15min":1},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":1.6339999999999999,"5min":1.6339999999999999,"15min":1.6339999999999999},"last":0.85399999999999998},{"interface":"front","average":{"1min":1.093,"5min":1.093,"15min":1.093},"min":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.704,"5min":1.704,"15min":1.704},"last":0.66300000000000003}]},{"osd":5,"last update":"Wed Sep 18 10:33:15 2024","interfaces":[{"interface":"back","average":{"1min":1.1699999999999999,"5min":1.1699999999999999,"15min":1.1699999999999999},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.9590000000000001,"5min":1.9590000000000001,"15min":1.9590000000000001},"last":0.999},{"interface":"front","average":{"1min":1.2050000000000001,"5min":1.2050000000000001,"15min":1.2050000000000001},"min":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"max":{"1min":2.206,"5min":2.206,"15min":2.206},"last":1.1200000000000001}]},{"osd":6,"last update":"Wed Sep 18 10:33:15 2024","interfaces":[{"interface":"back","average":{"1min":1.157,"5min":1.157,"15min":1.157},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":1.74,"5min":1.74,"15min":1.74},"last":1.0609999999999999},{"interface":"front","average":{"1min":1.135,"5min":1.135,"15min":1.135},"min":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"max":{"1min":2.8140000000000001,"5min":2.8140000000000001,"15min":2.8140000000000001},"last":1.1850000000000001}]},{"osd":7,"last update":"Wed Sep 18 10:33:15 2024","interfaces":[{"interface":"back","average":{"1min":1.268,"5min":1.268,"15min":1.268},"min":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"max":{"1min":2.004,"5min":2.004,"15min":2.004},"last":0.89900000000000002},{"interface":"front","average":{"1min":1.357,"5min":1.357,"15min":1.357},"min":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"max":{"1min":2.7250000000000001,"5min":2.7250000000000001,"15min":2.7250000000000001},"last":0.82299999999999995}]},{"osd":8,"last update":"Wed Sep 18 10:33:15 2024","interfaces":[{"interface":"back","average":{"1min":1.3049999999999999,"5min":1.3049999999999999,"15min":1.3049999999999999},"min":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":2.1419999999999999,"5min":2.1419999999999999,"15min":2.1419999999999999},"last":1.173},{"interface":"front","average":{"1min":1.3779999999999999,"5min":1.3779999999999999,"15min":1.3779999999999999},"min":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"max":{"1min":2.8980000000000001,"5min":2.8980000000000001,"15min":2.8980000000000001},"last":0.70299999999999996}]},{"osd":9,"last update":"Wed Sep 18 10:33:15 2024","interfaces":[{"interface":"back","average":{"1min":1.2989999999999999,"5min":1.2989999999999999,"15min":1.2989999999999999},"min":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"max":{"1min":2.3109999999999999,"5min":2.3109999999999999,"15min":2.3109999999999999},"last":1.262},{"interface":"front","average":{"1min":1.333,"5min":1.333,"15min":1.333},"min":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":2.2639999999999998,"5min":2.2639999999999998,"15min":2.2639999999999998},"last":1.077}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-18T10:34:14.761 INFO:tasks.ceph:pgid 1.7 last_scrub_stamp 2024-09-18T10:32:19.443095+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=10, tm_min=32, tm_sec=19, tm_wday=2, tm_yday=262, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=10, tm_min=34, tm_sec=8, tm_wday=2, tm_yday=262, tm_isdst=0) 2024-09-18T10:34:14.761 INFO:tasks.ceph:pgid 1.6 last_scrub_stamp 2024-09-18T10:32:19.443095+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=10, tm_min=32, tm_sec=19, tm_wday=2, tm_yday=262, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=10, tm_min=34, tm_sec=8, tm_wday=2, tm_yday=262, tm_isdst=0) 2024-09-18T10:34:14.761 INFO:tasks.ceph:pgid 1.5 last_scrub_stamp 2024-09-18T10:32:19.443095+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=10, tm_min=32, tm_sec=19, tm_wday=2, tm_yday=262, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=10, tm_min=34, tm_sec=8, tm_wday=2, tm_yday=262, tm_isdst=0) 2024-09-18T10:34:14.761 INFO:tasks.ceph:pgid 1.4 last_scrub_stamp 2024-09-18T10:32:19.443095+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=10, tm_min=32, tm_sec=19, tm_wday=2, tm_yday=262, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=10, tm_min=34, tm_sec=8, tm_wday=2, tm_yday=262, tm_isdst=0) 2024-09-18T10:34:14.761 INFO:tasks.ceph:pgid 1.3 last_scrub_stamp 2024-09-18T10:32:19.443095+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=10, tm_min=32, tm_sec=19, tm_wday=2, tm_yday=262, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=10, tm_min=34, tm_sec=8, tm_wday=2, tm_yday=262, tm_isdst=0) 2024-09-18T10:34:14.761 INFO:tasks.ceph:pgid 1.2 last_scrub_stamp 2024-09-18T10:32:19.443095+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=10, tm_min=32, tm_sec=19, tm_wday=2, tm_yday=262, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=10, tm_min=34, tm_sec=8, tm_wday=2, tm_yday=262, tm_isdst=0) 2024-09-18T10:34:14.761 INFO:tasks.ceph:pgid 1.1 last_scrub_stamp 2024-09-18T10:32:19.443095+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=10, tm_min=32, tm_sec=19, tm_wday=2, tm_yday=262, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=10, tm_min=34, tm_sec=8, tm_wday=2, tm_yday=262, tm_isdst=0) 2024-09-18T10:34:14.762 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-09-18T10:32:19.443095+0000 time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=10, tm_min=32, tm_sec=19, tm_wday=2, tm_yday=262, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=9, tm_mday=18, tm_hour=10, tm_min=34, tm_sec=8, tm_wday=2, tm_yday=262, tm_isdst=0) 2024-09-18T10:34:14.762 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-09-18T10:34:34.763 DEBUG:teuthology.orchestra.run.smithi134:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-18T10:34:35.066 INFO:teuthology.orchestra.run.smithi134.stdout: 2024-09-18T10:34:35.068 INFO:teuthology.orchestra.run.smithi134.stderr:dumped all 2024-09-18T10:34:35.079 INFO:teuthology.orchestra.run.smithi134.stdout:{"pg_ready":true,"pg_map":{"version":120,"stamp":"2024-09-18T10:34:33.439455+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5,"ondisk_log_size":5,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":10,"num_per_pool_osds":10,"num_per_pool_omap_osds":8,"kb":1048576000,"kb_used":2980272,"kb_used_data":6728,"kb_used_omap":16,"kb_used_meta":2973423,"kb_avail":1045595728,"statfs":{"total":1073741824000,"available":1070690025472,"internally_reserved":0,"allocated":6889472,"data_stored":3327748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":17040,"internal_metadata":3044785520},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003367"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":58,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-18T10:34:21.090252+0000","last_change":"2024-09-18T10:34:21.090190+0000","last_active":"2024-09-18T10:34:21.090252+0000","last_peered":"2024-09-18T10:34:21.090252+0000","last_clean":"2024-09-18T10:34:21.090252+0000","last_became_active":"2024-09-18T10:33:52.035591+0000","last_became_peered":"2024-09-18T10:33:52.035591+0000","last_unstale":"2024-09-18T10:34:21.090252+0000","last_undegraded":"2024-09-18T10:34:21.090252+0000","last_fullsized":"2024-09-18T10:34:21.090252+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T10:34:21.090022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:34:21.090022+0000","last_clean_scrub_stamp":"2024-09-18T10:34:21.090022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T18:54:40.030893+0000","scrub_duration":0.0030333600000000001,"objects_trimmed":0,"snaptrim_duration":0.00038275899999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":74,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-18T10:34:14.876289+0000","last_change":"2024-09-18T10:34:14.876226+0000","last_active":"2024-09-18T10:34:14.876289+0000","last_peered":"2024-09-18T10:34:14.876289+0000","last_clean":"2024-09-18T10:34:14.876289+0000","last_became_active":"2024-09-18T10:33:52.041895+0000","last_became_peered":"2024-09-18T10:33:52.041895+0000","last_unstale":"2024-09-18T10:34:14.876289+0000","last_undegraded":"2024-09-18T10:34:14.876289+0000","last_fullsized":"2024-09-18T10:34:14.876289+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T10:34:14.876045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:34:14.876045+0000","last_clean_scrub_stamp":"2024-09-18T10:34:14.876045+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T17:24:46.836899+0000","scrub_duration":0.0028866909999999998,"objects_trimmed":0,"snaptrim_duration":0.00034975400000000003,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":107,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-18T10:34:21.319398+0000","last_change":"2024-09-18T10:34:21.319342+0000","last_active":"2024-09-18T10:34:21.319398+0000","last_peered":"2024-09-18T10:34:21.319398+0000","last_clean":"2024-09-18T10:34:21.319398+0000","last_became_active":"2024-09-18T10:34:03.594428+0000","last_became_peered":"2024-09-18T10:34:03.594428+0000","last_unstale":"2024-09-18T10:34:21.319398+0000","last_undegraded":"2024-09-18T10:34:21.319398+0000","last_fullsized":"2024-09-18T10:34:21.319398+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T10:34:21.319177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:34:21.319177+0000","last_clean_scrub_stamp":"2024-09-18T10:34:21.319177+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:12:23.323859+0000","scrub_duration":0.0030135409999999998,"objects_trimmed":0,"snaptrim_duration":0.00048480000000000002,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0],"acting":[9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":72,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-18T10:34:14.112445+0000","last_change":"2024-09-18T10:34:14.112382+0000","last_active":"2024-09-18T10:34:14.112445+0000","last_peered":"2024-09-18T10:34:14.112445+0000","last_clean":"2024-09-18T10:34:14.112445+0000","last_became_active":"2024-09-18T10:34:03.531651+0000","last_became_peered":"2024-09-18T10:34:03.531651+0000","last_unstale":"2024-09-18T10:34:14.112445+0000","last_undegraded":"2024-09-18T10:34:14.112445+0000","last_fullsized":"2024-09-18T10:34:14.112445+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T10:34:14.112194+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:34:14.112194+0000","last_clean_scrub_stamp":"2024-09-18T10:34:14.112194+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T15:32:22.888639+0000","scrub_duration":0.0023741420000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.3","version":"60'2","reported_seq":94,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-18T10:34:20.108506+0000","last_change":"2024-09-18T10:34:20.108437+0000","last_active":"2024-09-18T10:34:20.108506+0000","last_peered":"2024-09-18T10:34:20.108506+0000","last_clean":"2024-09-18T10:34:20.108506+0000","last_became_active":"2024-09-18T10:34:03.137466+0000","last_became_peered":"2024-09-18T10:34:03.137466+0000","last_unstale":"2024-09-18T10:34:20.108506+0000","last_undegraded":"2024-09-18T10:34:20.108506+0000","last_fullsized":"2024-09-18T10:34:20.108506+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"60'2","last_scrub_stamp":"2024-09-18T10:34:20.108282+0000","last_deep_scrub":"60'2","last_deep_scrub_stamp":"2024-09-18T10:34:20.108282+0000","last_clean_scrub_stamp":"2024-09-18T10:34:20.108282+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:12:22.112964+0000","scrub_duration":0.0055851750000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.2","version":"60'3","reported_seq":97,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-18T10:34:25.475817+0000","last_change":"2024-09-18T10:34:25.475739+0000","last_active":"2024-09-18T10:34:25.475817+0000","last_peered":"2024-09-18T10:34:25.475817+0000","last_clean":"2024-09-18T10:34:25.475817+0000","last_became_active":"2024-09-18T10:34:03.136418+0000","last_became_peered":"2024-09-18T10:34:03.136418+0000","last_unstale":"2024-09-18T10:34:25.475817+0000","last_undegraded":"2024-09-18T10:34:25.475817+0000","last_fullsized":"2024-09-18T10:34:25.475817+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"60'3","last_scrub_stamp":"2024-09-18T10:34:25.475585+0000","last_deep_scrub":"60'3","last_deep_scrub_stamp":"2024-09-18T10:34:25.475585+0000","last_clean_scrub_stamp":"2024-09-18T10:34:25.475585+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T11:12:27.480267+0000","scrub_duration":0.0055155739999999997,"objects_trimmed":0,"snaptrim_duration":0.0019475040000000001,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":55,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-18T10:34:16.653674+0000","last_change":"2024-09-18T10:34:16.653615+0000","last_active":"2024-09-18T10:34:16.653674+0000","last_peered":"2024-09-18T10:34:16.653674+0000","last_clean":"2024-09-18T10:34:16.653674+0000","last_became_active":"2024-09-18T10:33:52.038239+0000","last_became_peered":"2024-09-18T10:33:52.038239+0000","last_unstale":"2024-09-18T10:34:16.653674+0000","last_undegraded":"2024-09-18T10:34:16.653674+0000","last_fullsized":"2024-09-18T10:34:16.653674+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T10:34:16.653447+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:34:16.653447+0000","last_clean_scrub_stamp":"2024-09-18T10:34:16.653447+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T10:36:55.565621+0000","scrub_duration":0.002990861,"objects_trimmed":0,"snaptrim_duration":0.00062722600000000004,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":74,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-18T10:34:13.787735+0000","last_change":"2024-09-18T10:34:13.787711+0000","last_active":"2024-09-18T10:34:13.787735+0000","last_peered":"2024-09-18T10:34:13.787735+0000","last_clean":"2024-09-18T10:34:13.787735+0000","last_became_active":"2024-09-18T10:33:52.037028+0000","last_became_peered":"2024-09-18T10:33:52.037028+0000","last_unstale":"2024-09-18T10:34:13.787735+0000","last_undegraded":"2024-09-18T10:34:13.787735+0000","last_fullsized":"2024-09-18T10:34:13.787735+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-18T10:34:13.787643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-18T10:34:13.787643+0000","last_clean_scrub_stamp":"2024-09-18T10:34:13.787643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-19T14:16:37.909456+0000","scrub_duration":0.0020430299999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9],"acting":[5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":19,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5,"ondisk_log_size":5,"up":16,"acting":16,"num_store_stats":9}],"osd_stats":[{"osd":9,"up_from":41,"seq":176093659165,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":298024,"kb_used_data":672,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":104559576,"statfs":{"total":107374182400,"available":107069005824,"internally_reserved":0,"allocated":688128,"data_stored":332771,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,6,7,8],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 10:33:18 2024","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":1.1930000000000001,"5min":1.1930000000000001,"15min":1.1930000000000001},"last":0.626},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.3029999999999999,"5min":1.3029999999999999,"15min":1.3029999999999999},"last":1.0469999999999999}]},{"osd":1,"last update":"Wed Sep 18 10:33:18 2024","interfaces":[{"interface":"back","average":{"1min":0.873,"5min":0.873,"15min":0.873},"min":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.3919999999999999,"5min":1.3919999999999999,"15min":1.3919999999999999},"last":0.746},{"interface":"front","average":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":2.4359999999999999,"5min":2.4359999999999999,"15min":2.4359999999999999},"last":0.995}]},{"osd":2,"last update":"Wed Sep 18 10:33:18 2024","interfaces":[{"interface":"back","average":{"1min":1.03,"5min":1.03,"15min":1.03},"min":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":1.526,"5min":1.526,"15min":1.526},"last":0.88900000000000001},{"interface":"front","average":{"1min":1.044,"5min":1.044,"15min":1.044},"min":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"max":{"1min":1.591,"5min":1.591,"15min":1.591},"last":0.81999999999999995}]},{"osd":3,"last update":"Wed Sep 18 10:33:18 2024","interfaces":[{"interface":"back","average":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.542,"5min":1.542,"15min":1.542},"last":1.341},{"interface":"front","average":{"1min":1.274,"5min":1.274,"15min":1.274},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.8440000000000001,"5min":1.8440000000000001,"15min":1.8440000000000001},"last":1.198}]},{"osd":4,"last update":"Wed Sep 18 10:33:18 2024","interfaces":[{"interface":"back","average":{"1min":1.2430000000000001,"5min":1.2430000000000001,"15min":1.2430000000000001},"min":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"max":{"1min":1.8180000000000001,"5min":1.8180000000000001,"15min":1.8180000000000001},"last":1.1140000000000001},{"interface":"front","average":{"1min":1.276,"5min":1.276,"15min":1.276},"min":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"max":{"1min":1.681,"5min":1.681,"15min":1.681},"last":1.2250000000000001}]},{"osd":5,"last update":"Wed Sep 18 10:33:18 2024","interfaces":[{"interface":"back","average":{"1min":1.3140000000000001,"5min":1.3140000000000001,"15min":1.3140000000000001},"min":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"max":{"1min":1.8600000000000001,"5min":1.8600000000000001,"15min":1.8600000000000001},"last":1.306},{"interface":"front","average":{"1min":1.3009999999999999,"5min":1.3009999999999999,"15min":1.3009999999999999},"min":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"max":{"1min":1.7090000000000001,"5min":1.7090000000000001,"15min":1.7090000000000001},"last":1.5589999999999999}]},{"osd":6,"last update":"Wed Sep 18 10:33:18 2024","interfaces":[{"interface":"back","average":{"1min":1.3029999999999999,"5min":1.3029999999999999,"15min":1.3029999999999999},"min":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"max":{"1min":2.778,"5min":2.778,"15min":2.778},"last":1.6439999999999999},{"interface":"front","average":{"1min":1.345,"5min":1.345,"15min":1.345},"min":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":1.921,"5min":1.921,"15min":1.921},"last":1.27}]},{"osd":7,"last update":"Wed Sep 18 10:33:18 2024","interfaces":[{"interface":"back","average":{"1min":1.3340000000000001,"5min":1.3340000000000001,"15min":1.3340000000000001},"min":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"max":{"1min":1.6830000000000001,"5min":1.6830000000000001,"15min":1.6830000000000001},"last":1.405},{"interface":"front","average":{"1min":1.417,"5min":1.417,"15min":1.417},"min":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"max":{"1min":3.0390000000000001,"5min":3.0390000000000001,"15min":3.0390000000000001},"last":1.458}]},{"osd":8,"last update":"Wed Sep 18 10:33:18 2024","interfaces":[{"interface":"back","average":{"1min":1.4330000000000001,"5min":1.4330000000000001,"15min":1.4330000000000001},"min":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"max":{"1min":2.8500000000000001,"5min":2.8500000000000001,"15min":2.8500000000000001},"last":1.704},{"interface":"front","average":{"1min":1.4079999999999999,"5min":1.4079999999999999,"15min":1.4079999999999999},"min":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"max":{"1min":2.0070000000000001,"5min":2.0070000000000001,"15min":2.0070000000000001},"last":1.512}]}]},{"osd":8,"up_from":48,"seq":206158430238,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":104857600,"kb_used":298032,"kb_used_data":672,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":104559568,"statfs":{"total":107374182400,"available":107068997632,"internally_reserved":0,"allocated":688128,"data_stored":332771,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,6,7,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 10:33:16 2024","interfaces":[{"interface":"back","average":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":1.26,"5min":1.26,"15min":1.26},"last":1.3680000000000001},{"interface":"front","average":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":1.238,"5min":1.238,"15min":1.238},"last":0.76600000000000001}]},{"osd":1,"last update":"Wed Sep 18 10:33:16 2024","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.751,"15min":0.751},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":1.129,"5min":1.129,"15min":1.129},"last":0.98599999999999999},{"interface":"front","average":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"min":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"max":{"1min":1.2390000000000001,"5min":1.2390000000000001,"15min":1.2390000000000001},"last":0.66800000000000004}]},{"osd":2,"last update":"Wed Sep 18 10:33:16 2024","interfaces":[{"interface":"back","average":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":1.6439999999999999,"5min":1.6439999999999999,"15min":1.6439999999999999},"last":0.93100000000000005},{"interface":"front","average":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":1.2529999999999999,"5min":1.2529999999999999,"15min":1.2529999999999999},"last":0.85399999999999998}]},{"osd":3,"last update":"Wed Sep 18 10:33:16 2024","interfaces":[{"interface":"back","average":{"1min":1.109,"5min":1.109,"15min":1.109},"min":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"max":{"1min":1.538,"5min":1.538,"15min":1.538},"last":1.536},{"interface":"front","average":{"1min":1.0389999999999999,"5min":1.0389999999999999,"15min":1.0389999999999999},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.4810000000000001,"5min":1.4810000000000001,"15min":1.4810000000000001},"last":1.1599999999999999}]},{"osd":4,"last update":"Wed Sep 18 10:33:16 2024","interfaces":[{"interface":"back","average":{"1min":1.0620000000000001,"5min":1.0620000000000001,"15min":1.0620000000000001},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.8169999999999999,"5min":1.8169999999999999,"15min":1.8169999999999999},"last":1.2390000000000001},{"interface":"front","average":{"1min":1.2549999999999999,"5min":1.2549999999999999,"15min":1.2549999999999999},"min":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"max":{"1min":2.1480000000000001,"5min":2.1480000000000001,"15min":2.1480000000000001},"last":1.3999999999999999}]},{"osd":5,"last update":"Wed Sep 18 10:33:16 2024","interfaces":[{"interface":"back","average":{"1min":1.216,"5min":1.216,"15min":1.216},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":2.3109999999999999,"5min":2.3109999999999999,"15min":2.3109999999999999},"last":1.5089999999999999},{"interface":"front","average":{"1min":1.161,"5min":1.161,"15min":1.161},"min":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":2.0169999999999999,"5min":2.0169999999999999,"15min":2.0169999999999999},"last":1.4430000000000001}]},{"osd":6,"last update":"Wed Sep 18 10:33:16 2024","interfaces":[{"interface":"back","average":{"1min":1.2889999999999999,"5min":1.2889999999999999,"15min":1.2889999999999999},"min":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"max":{"1min":2.1899999999999999,"5min":2.1899999999999999,"15min":2.1899999999999999},"last":1.603},{"interface":"front","average":{"1min":1.1799999999999999,"5min":1.1799999999999999,"15min":1.1799999999999999},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":2.069,"5min":2.069,"15min":2.069},"last":1.476}]},{"osd":7,"last update":"Wed Sep 18 10:33:16 2024","interfaces":[{"interface":"back","average":{"1min":1.2589999999999999,"5min":1.2589999999999999,"15min":1.2589999999999999},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":2.2130000000000001,"5min":2.2130000000000001,"15min":2.2130000000000001},"last":1.6180000000000001},{"interface":"front","average":{"1min":1.4039999999999999,"5min":1.4039999999999999,"15min":1.4039999999999999},"min":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"max":{"1min":2.2450000000000001,"5min":2.2450000000000001,"15min":2.2450000000000001},"last":1.6439999999999999}]},{"osd":9,"last update":"Wed Sep 18 10:33:16 2024","interfaces":[{"interface":"back","average":{"1min":1.304,"5min":1.304,"15min":1.304},"min":{"1min":0.63,"5min":0.63,"15min":0.63},"max":{"1min":2.1800000000000002,"5min":2.1800000000000002,"15min":2.1800000000000002},"last":1.304},{"interface":"front","average":{"1min":1.3440000000000001,"5min":1.3440000000000001,"15min":1.3440000000000001},"min":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"max":{"1min":2.3969999999999998,"5min":2.3969999999999998,"15min":2.3969999999999998},"last":1.575}]}]},{"osd":7,"up_from":48,"seq":206158430237,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":298024,"kb_used_data":672,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":104559576,"statfs":{"total":107374182400,"available":107069005824,"internally_reserved":0,"allocated":688128,"data_stored":332771,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,6,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 10:33:17 2024","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.202,"5min":1.202,"15min":1.202},"last":0.81200000000000006},{"interface":"front","average":{"1min":0.75,"5min":0.75,"15min":0.75},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":1.486,"5min":1.486,"15min":1.486},"last":1.2849999999999999}]},{"osd":1,"last update":"Wed Sep 18 10:33:17 2024","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.3009999999999999,"5min":1.3009999999999999,"15min":1.3009999999999999},"last":0.872},{"interface":"front","average":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"max":{"1min":1.8480000000000001,"5min":1.8480000000000001,"15min":1.8480000000000001},"last":0.91800000000000004}]},{"osd":2,"last update":"Wed Sep 18 10:33:17 2024","interfaces":[{"interface":"back","average":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":1.5980000000000001,"5min":1.5980000000000001,"15min":1.5980000000000001},"last":1.1060000000000001},{"interface":"front","average":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.3440000000000001,"5min":1.3440000000000001,"15min":1.3440000000000001},"last":1.04}]},{"osd":3,"last update":"Wed Sep 18 10:33:17 2024","interfaces":[{"interface":"back","average":{"1min":1.1000000000000001,"5min":1.1000000000000001,"15min":1.1000000000000001},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.9319999999999999,"5min":1.9319999999999999,"15min":1.9319999999999999},"last":1.3069999999999999},{"interface":"front","average":{"1min":1.0720000000000001,"5min":1.0720000000000001,"15min":1.0720000000000001},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.8879999999999999,"5min":1.8879999999999999,"15min":1.8879999999999999},"last":1.4359999999999999}]},{"osd":4,"last update":"Wed Sep 18 10:33:17 2024","interfaces":[{"interface":"back","average":{"1min":1.097,"5min":1.097,"15min":1.097},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":2.1379999999999999,"5min":2.1379999999999999,"15min":2.1379999999999999},"last":1.069},{"interface":"front","average":{"1min":1.1060000000000001,"5min":1.1060000000000001,"15min":1.1060000000000001},"min":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":2.0110000000000001,"5min":2.0110000000000001,"15min":2.0110000000000001},"last":1.236}]},{"osd":5,"last update":"Wed Sep 18 10:33:17 2024","interfaces":[{"interface":"back","average":{"1min":1.194,"5min":1.194,"15min":1.194},"min":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"max":{"1min":2.4889999999999999,"5min":2.4889999999999999,"15min":2.4889999999999999},"last":1.198},{"interface":"front","average":{"1min":1.2230000000000001,"5min":1.2230000000000001,"15min":1.2230000000000001},"min":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"max":{"1min":2.4289999999999998,"5min":2.4289999999999998,"15min":2.4289999999999998},"last":1.157}]},{"osd":6,"last update":"Wed Sep 18 10:33:17 2024","interfaces":[{"interface":"back","average":{"1min":1.2769999999999999,"5min":1.2769999999999999,"15min":1.2769999999999999},"min":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"max":{"1min":2.0950000000000002,"5min":2.0950000000000002,"15min":2.0950000000000002},"last":1.5609999999999999},{"interface":"front","average":{"1min":1.218,"5min":1.218,"15min":1.218},"min":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":1.9450000000000001,"5min":1.9450000000000001,"15min":1.9450000000000001},"last":1.7270000000000001}]},{"osd":8,"last update":"Wed Sep 18 10:33:17 2024","interfaces":[{"interface":"back","average":{"1min":1.2969999999999999,"5min":1.2969999999999999,"15min":1.2969999999999999},"min":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"max":{"1min":2.6389999999999998,"5min":2.6389999999999998,"15min":2.6389999999999998},"last":1.6140000000000001},{"interface":"front","average":{"1min":1.3600000000000001,"5min":1.3600000000000001,"15min":1.3600000000000001},"min":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"max":{"1min":2.1520000000000001,"5min":2.1520000000000001,"15min":2.1520000000000001},"last":1.506}]},{"osd":9,"last update":"Wed Sep 18 10:33:17 2024","interfaces":[{"interface":"back","average":{"1min":1.2470000000000001,"5min":1.2470000000000001,"15min":1.2470000000000001},"min":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":2.0680000000000001,"5min":2.0680000000000001,"15min":2.0680000000000001},"last":1.375},{"interface":"front","average":{"1min":1.282,"5min":1.282,"15min":1.282},"min":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"max":{"1min":1.8460000000000001,"5min":1.8460000000000001,"15min":1.8460000000000001},"last":1.464}]}]},{"osd":6,"up_from":48,"seq":206158430237,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":298024,"kb_used_data":672,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":104559576,"statfs":{"total":107374182400,"available":107069005824,"internally_reserved":0,"allocated":688128,"data_stored":332771,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,7,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 10:33:17 2024","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":1.073,"5min":1.073,"15min":1.073},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":1.3720000000000001,"5min":1.3720000000000001,"15min":1.3720000000000001},"last":0.92000000000000004}]},{"osd":1,"last update":"Wed Sep 18 10:33:17 2024","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.751,"15min":0.751},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":1.6599999999999999,"5min":1.6599999999999999,"15min":1.6599999999999999},"last":0.84899999999999998},{"interface":"front","average":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":2.0459999999999998,"5min":2.0459999999999998,"15min":2.0459999999999998},"last":1.075}]},{"osd":2,"last update":"Wed Sep 18 10:33:17 2024","interfaces":[{"interface":"back","average":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":1.1399999999999999,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":0.77500000000000002},{"interface":"front","average":{"1min":1.052,"5min":1.052,"15min":1.052},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":1.907,"5min":1.907,"15min":1.907},"last":0.878}]},{"osd":3,"last update":"Wed Sep 18 10:33:17 2024","interfaces":[{"interface":"back","average":{"1min":1.0429999999999999,"5min":1.0429999999999999,"15min":1.0429999999999999},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.7490000000000001,"5min":1.7490000000000001,"15min":1.7490000000000001},"last":0.96899999999999997},{"interface":"front","average":{"1min":1.1830000000000001,"5min":1.1830000000000001,"15min":1.1830000000000001},"min":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"max":{"1min":1.8680000000000001,"5min":1.8680000000000001,"15min":1.8680000000000001},"last":0.72199999999999998}]},{"osd":4,"last update":"Wed Sep 18 10:33:17 2024","interfaces":[{"interface":"back","average":{"1min":1.2070000000000001,"5min":1.2070000000000001,"15min":1.2070000000000001},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.976,"5min":1.976,"15min":1.976},"last":0.80700000000000005},{"interface":"front","average":{"1min":1.127,"5min":1.127,"15min":1.127},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":2.544,"5min":2.544,"15min":2.544},"last":1.7010000000000001}]},{"osd":5,"last update":"Wed Sep 18 10:33:17 2024","interfaces":[{"interface":"back","average":{"1min":1.2170000000000001,"5min":1.2170000000000001,"15min":1.2170000000000001},"min":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"max":{"1min":2.1539999999999999,"5min":2.1539999999999999,"15min":2.1539999999999999},"last":1.6499999999999999},{"interface":"front","average":{"1min":1.296,"5min":1.296,"15min":1.296},"min":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":2.2469999999999999,"5min":2.2469999999999999,"15min":2.2469999999999999},"last":1.478}]},{"osd":7,"last update":"Wed Sep 18 10:33:17 2024","interfaces":[{"interface":"back","average":{"1min":1.246,"5min":1.246,"15min":1.246},"min":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":2.6440000000000001,"5min":2.6440000000000001,"15min":2.6440000000000001},"last":1.238},{"interface":"front","average":{"1min":1.7609999999999999,"5min":1.7609999999999999,"15min":1.7609999999999999},"min":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"max":{"1min":10.385,"5min":10.385,"15min":10.385},"last":1.4510000000000001}]},{"osd":8,"last update":"Wed Sep 18 10:33:17 2024","interfaces":[{"interface":"back","average":{"1min":1.3640000000000001,"5min":1.3640000000000001,"15min":1.3640000000000001},"min":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"max":{"1min":2.444,"5min":2.444,"15min":2.444},"last":1.7569999999999999},{"interface":"front","average":{"1min":1.274,"5min":1.274,"15min":1.274},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":2.714,"5min":2.714,"15min":2.714},"last":1.1639999999999999}]},{"osd":9,"last update":"Wed Sep 18 10:33:17 2024","interfaces":[{"interface":"back","average":{"1min":1.2789999999999999,"5min":1.2789999999999999,"15min":1.2789999999999999},"min":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"max":{"1min":2.2879999999999998,"5min":2.2879999999999998,"15min":2.2879999999999998},"last":1.51},{"interface":"front","average":{"1min":1.3240000000000001,"5min":1.3240000000000001,"15min":1.3240000000000001},"min":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":2.5960000000000001,"5min":2.5960000000000001,"15min":2.5960000000000001},"last":1.5700000000000001}]}]},{"osd":5,"up_from":48,"seq":206158430239,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":298024,"kb_used_data":672,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":104559576,"statfs":{"total":107374182400,"available":107069005824,"internally_reserved":0,"allocated":688128,"data_stored":332771,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,6,7,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 10:33:18 2024","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.111,"5min":1.111,"15min":1.111},"last":0.90600000000000003},{"interface":"front","average":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"min":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.5249999999999999,"5min":1.5249999999999999,"15min":1.5249999999999999},"last":1.077}]},{"osd":1,"last update":"Wed Sep 18 10:33:18 2024","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.173,"5min":1.173,"15min":1.173},"last":0.82399999999999995},{"interface":"front","average":{"1min":0.95699999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.8280000000000001,"5min":1.8280000000000001,"15min":1.8280000000000001},"last":1.411}]},{"osd":2,"last update":"Wed Sep 18 10:33:18 2024","interfaces":[{"interface":"back","average":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"min":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":1.4159999999999999,"5min":1.4159999999999999,"15min":1.4159999999999999},"last":1.034},{"interface":"front","average":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"min":{"1min":0.627,"5min":0.627,"15min":0.627},"max":{"1min":2.3849999999999998,"5min":2.3849999999999998,"15min":2.3849999999999998},"last":1.1240000000000001}]},{"osd":3,"last update":"Wed Sep 18 10:33:18 2024","interfaces":[{"interface":"back","average":{"1min":1.079,"5min":1.079,"15min":1.079},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.923,"5min":1.923,"15min":1.923},"last":1.3680000000000001},{"interface":"front","average":{"1min":1.268,"5min":1.268,"15min":1.268},"min":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"max":{"1min":2.073,"5min":2.073,"15min":2.073},"last":1.0029999999999999}]},{"osd":4,"last update":"Wed Sep 18 10:33:18 2024","interfaces":[{"interface":"back","average":{"1min":1.123,"5min":1.123,"15min":1.123},"min":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"max":{"1min":2.141,"5min":2.141,"15min":2.141},"last":0.85699999999999998},{"interface":"front","average":{"1min":1.2490000000000001,"5min":1.2490000000000001,"15min":1.2490000000000001},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":2.3300000000000001,"5min":2.3300000000000001,"15min":2.3300000000000001},"last":1.1799999999999999}]},{"osd":6,"last update":"Wed Sep 18 10:33:18 2024","interfaces":[{"interface":"back","average":{"1min":1.2769999999999999,"5min":1.2769999999999999,"15min":1.2769999999999999},"min":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"max":{"1min":1.806,"5min":1.806,"15min":1.806},"last":1.2270000000000001},{"interface":"front","average":{"1min":1.274,"5min":1.274,"15min":1.274},"min":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"max":{"1min":2.6120000000000001,"5min":2.6120000000000001,"15min":2.6120000000000001},"last":0.96299999999999997}]},{"osd":7,"last update":"Wed Sep 18 10:33:18 2024","interfaces":[{"interface":"back","average":{"1min":1.3380000000000001,"5min":1.3380000000000001,"15min":1.3380000000000001},"min":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"max":{"1min":2.661,"5min":2.661,"15min":2.661},"last":1.2889999999999999},{"interface":"front","average":{"1min":1.3819999999999999,"5min":1.3819999999999999,"15min":1.3819999999999999},"min":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"max":{"1min":2.548,"5min":2.548,"15min":2.548},"last":1.3260000000000001}]},{"osd":8,"last update":"Wed Sep 18 10:33:18 2024","interfaces":[{"interface":"back","average":{"1min":1.359,"5min":1.359,"15min":1.359},"min":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"max":{"1min":2.7080000000000002,"5min":2.7080000000000002,"15min":2.7080000000000002},"last":1.466},{"interface":"front","average":{"1min":1.327,"5min":1.327,"15min":1.327},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":2.4399999999999999,"5min":2.4399999999999999,"15min":2.4399999999999999},"last":1.4410000000000001}]},{"osd":9,"last update":"Wed Sep 18 10:33:18 2024","interfaces":[{"interface":"back","average":{"1min":1.3859999999999999,"5min":1.3859999999999999,"15min":1.3859999999999999},"min":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"max":{"1min":2.2130000000000001,"5min":2.2130000000000001,"15min":2.2130000000000001},"last":1.5569999999999999},{"interface":"front","average":{"1min":1.4019999999999999,"5min":1.4019999999999999,"15min":1.4019999999999999},"min":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"max":{"1min":2.782,"5min":2.782,"15min":2.782},"last":1.6100000000000001}]}]},{"osd":4,"up_from":48,"seq":206158430237,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":298028,"kb_used_data":676,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":104559572,"statfs":{"total":107374182400,"available":107069001728,"internally_reserved":0,"allocated":692224,"data_stored":332790,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,5,6,7,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 10:33:16 2024","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":1.0960000000000001,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.62},{"interface":"front","average":{"1min":0.745,"5min":0.745,"15min":0.745},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.171,"5min":1.171,"15min":1.171},"last":0.81399999999999995}]},{"osd":1,"last update":"Wed Sep 18 10:33:16 2024","interfaces":[{"interface":"back","average":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":1.1579999999999999,"5min":1.1579999999999999,"15min":1.1579999999999999},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.583,"5min":1.583,"15min":1.583},"last":1.0680000000000001}]},{"osd":2,"last update":"Wed Sep 18 10:33:16 2024","interfaces":[{"interface":"back","average":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":2.1139999999999999,"5min":2.1139999999999999,"15min":2.1139999999999999},"last":0.96899999999999997},{"interface":"front","average":{"1min":0.995,"5min":0.995,"15min":0.995},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.5660000000000001,"5min":1.5660000000000001,"15min":1.5660000000000001},"last":0.57999999999999996}]},{"osd":3,"last update":"Wed Sep 18 10:33:16 2024","interfaces":[{"interface":"back","average":{"1min":1.0620000000000001,"5min":1.0620000000000001,"15min":1.0620000000000001},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.6200000000000001,"5min":1.6200000000000001,"15min":1.6200000000000001},"last":0.75},{"interface":"front","average":{"1min":1.1699999999999999,"5min":1.1699999999999999,"15min":1.1699999999999999},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.732,"5min":1.732,"15min":1.732},"last":1.512}]},{"osd":5,"last update":"Wed Sep 18 10:33:16 2024","interfaces":[{"interface":"back","average":{"1min":1.159,"5min":1.159,"15min":1.159},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":2.2509999999999999,"5min":2.2509999999999999,"15min":2.2509999999999999},"last":1.454},{"interface":"front","average":{"1min":1.1830000000000001,"5min":1.1830000000000001,"15min":1.1830000000000001},"min":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"max":{"1min":1.873,"5min":1.873,"15min":1.873},"last":1.1020000000000001}]},{"osd":6,"last update":"Wed Sep 18 10:33:16 2024","interfaces":[{"interface":"back","average":{"1min":1.214,"5min":1.214,"15min":1.214},"min":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"max":{"1min":2.2879999999999998,"5min":2.2879999999999998,"15min":2.2879999999999998},"last":1.206},{"interface":"front","average":{"1min":1.25,"5min":1.25,"15min":1.25},"min":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":2.661,"5min":2.661,"15min":2.661},"last":1.595}]},{"osd":7,"last update":"Wed Sep 18 10:33:16 2024","interfaces":[{"interface":"back","average":{"1min":1.319,"5min":1.319,"15min":1.319},"min":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"max":{"1min":1.895,"5min":1.895,"15min":1.895},"last":1.012},{"interface":"front","average":{"1min":1.2450000000000001,"5min":1.2450000000000001,"15min":1.2450000000000001},"min":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":2.0579999999999998,"5min":2.0579999999999998,"15min":2.0579999999999998},"last":1.1519999999999999}]},{"osd":8,"last update":"Wed Sep 18 10:33:16 2024","interfaces":[{"interface":"back","average":{"1min":1.278,"5min":1.278,"15min":1.278},"min":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"max":{"1min":2.173,"5min":2.173,"15min":2.173},"last":1.5549999999999999},{"interface":"front","average":{"1min":1.3240000000000001,"5min":1.3240000000000001,"15min":1.3240000000000001},"min":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":1.9750000000000001,"5min":1.9750000000000001,"15min":1.9750000000000001},"last":1.401}]},{"osd":9,"last update":"Wed Sep 18 10:33:16 2024","interfaces":[{"interface":"back","average":{"1min":1.327,"5min":1.327,"15min":1.327},"min":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":2.448,"5min":2.448,"15min":2.448},"last":1.3140000000000001},{"interface":"front","average":{"1min":1.389,"5min":1.389,"15min":1.389},"min":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":0.98999999999999999},"max":{"1min":2.242,"5min":2.242,"15min":2.242},"last":0.93300000000000005}]}]},{"osd":3,"up_from":48,"seq":206158430239,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":298024,"kb_used_data":672,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":104559576,"statfs":{"total":107374182400,"available":107069005824,"internally_reserved":0,"allocated":688128,"data_stored":332771,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,4,5,6,7,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 10:33:15 2024","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.095,"5min":1.095,"15min":1.095},"last":0.89200000000000002},{"interface":"front","average":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"min":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":2.121,"5min":2.121,"15min":2.121},"last":0.85199999999999998}]},{"osd":1,"last update":"Wed Sep 18 10:33:15 2024","interfaces":[{"interface":"back","average":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":1.9259999999999999,"5min":1.9259999999999999,"15min":1.9259999999999999},"last":1.127},{"interface":"front","average":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"min":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":1.2090000000000001,"5min":1.2090000000000001,"15min":1.2090000000000001},"last":1.421}]},{"osd":2,"last update":"Wed Sep 18 10:33:15 2024","interfaces":[{"interface":"back","average":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.204,"5min":1.204,"15min":1.204},"last":1.171},{"interface":"front","average":{"1min":1.1870000000000001,"5min":1.1870000000000001,"15min":1.1870000000000001},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":3.2040000000000002,"5min":3.2040000000000002,"15min":3.2040000000000002},"last":0.627}]},{"osd":4,"last update":"Wed Sep 18 10:33:15 2024","interfaces":[{"interface":"back","average":{"1min":1.071,"5min":1.071,"15min":1.071},"min":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":1.49,"5min":1.49,"15min":1.49},"last":0.71499999999999997},{"interface":"front","average":{"1min":1.266,"5min":1.266,"15min":1.266},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":2.7519999999999998,"5min":2.7519999999999998,"15min":2.7519999999999998},"last":1.306}]},{"osd":5,"last update":"Wed Sep 18 10:33:15 2024","interfaces":[{"interface":"back","average":{"1min":1.1830000000000001,"5min":1.1830000000000001,"15min":1.1830000000000001},"min":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"max":{"1min":2.0630000000000002,"5min":2.0630000000000002,"15min":2.0630000000000002},"last":0.78100000000000003},{"interface":"front","average":{"1min":1.1259999999999999,"5min":1.1259999999999999,"15min":1.1259999999999999},"min":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"max":{"1min":1.5149999999999999,"5min":1.5149999999999999,"15min":1.5149999999999999},"last":1.256}]},{"osd":6,"last update":"Wed Sep 18 10:33:15 2024","interfaces":[{"interface":"back","average":{"1min":1.3620000000000001,"5min":1.3620000000000001,"15min":1.3620000000000001},"min":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":4.1029999999999998,"5min":4.1029999999999998,"15min":4.1029999999999998},"last":1.5609999999999999},{"interface":"front","average":{"1min":1.268,"5min":1.268,"15min":1.268},"min":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"max":{"1min":2.2789999999999999,"5min":2.2789999999999999,"15min":2.2789999999999999},"last":1.032}]},{"osd":7,"last update":"Wed Sep 18 10:33:15 2024","interfaces":[{"interface":"back","average":{"1min":1.2010000000000001,"5min":1.2010000000000001,"15min":1.2010000000000001},"min":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"max":{"1min":2.1880000000000002,"5min":2.1880000000000002,"15min":2.1880000000000002},"last":1.2030000000000001},{"interface":"front","average":{"1min":1.282,"5min":1.282,"15min":1.282},"min":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"max":{"1min":3.0409999999999999,"5min":3.0409999999999999,"15min":3.0409999999999999},"last":1.4930000000000001}]},{"osd":8,"last update":"Wed Sep 18 10:33:15 2024","interfaces":[{"interface":"back","average":{"1min":1.5109999999999999,"5min":1.5109999999999999,"15min":1.5109999999999999},"min":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"max":{"1min":3.5590000000000002,"5min":3.5590000000000002,"15min":3.5590000000000002},"last":1.6080000000000001},{"interface":"front","average":{"1min":1.3540000000000001,"5min":1.3540000000000001,"15min":1.3540000000000001},"min":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"max":{"1min":2.3919999999999999,"5min":2.3919999999999999,"15min":2.3919999999999999},"last":1.6599999999999999}]},{"osd":9,"last update":"Wed Sep 18 10:33:15 2024","interfaces":[{"interface":"back","average":{"1min":1.379,"5min":1.379,"15min":1.379},"min":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"max":{"1min":3.1259999999999999,"5min":3.1259999999999999,"15min":3.1259999999999999},"last":1.5349999999999999},{"interface":"front","average":{"1min":1.5109999999999999,"5min":1.5109999999999999,"15min":1.5109999999999999},"min":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"max":{"1min":3.6339999999999999,"5min":3.6339999999999999,"15min":3.6339999999999999},"last":1.4610000000000001}]}]},{"osd":2,"up_from":48,"seq":206158430237,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":104857600,"kb_used":298028,"kb_used_data":672,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":104559572,"statfs":{"total":107374182400,"available":107069001728,"internally_reserved":0,"allocated":688128,"data_stored":332771,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,3,4,5,6,7,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 10:33:18 2024","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1.109,"5min":1.109,"15min":1.109},"last":0.90000000000000002},{"interface":"front","average":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":1.5029999999999999,"5min":1.5029999999999999,"15min":1.5029999999999999},"last":0.72099999999999997}]},{"osd":1,"last update":"Wed Sep 18 10:33:18 2024","interfaces":[{"interface":"back","average":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"min":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.651,"5min":1.651,"15min":1.651},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":1.1559999999999999,"5min":1.1559999999999999,"15min":1.1559999999999999},"last":1.2190000000000001}]},{"osd":3,"last update":"Wed Sep 18 10:33:18 2024","interfaces":[{"interface":"back","average":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":1.2869999999999999,"5min":1.2869999999999999,"15min":1.2869999999999999},"last":0.80500000000000005},{"interface":"front","average":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.7390000000000001,"5min":1.7390000000000001,"15min":1.7390000000000001},"last":1.349}]},{"osd":4,"last update":"Wed Sep 18 10:33:18 2024","interfaces":[{"interface":"back","average":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"min":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"max":{"1min":1.333,"5min":1.333,"15min":1.333},"last":0.84899999999999998},{"interface":"front","average":{"1min":1.151,"5min":1.151,"15min":1.151},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.7869999999999999,"5min":1.7869999999999999,"15min":1.7869999999999999},"last":0.98699999999999999}]},{"osd":5,"last update":"Wed Sep 18 10:33:18 2024","interfaces":[{"interface":"back","average":{"1min":1.1100000000000001,"5min":1.1100000000000001,"15min":1.1100000000000001},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.597,"5min":1.597,"15min":1.597},"last":1.1220000000000001},{"interface":"front","average":{"1min":1.246,"5min":1.246,"15min":1.246},"min":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"max":{"1min":1.873,"5min":1.873,"15min":1.873},"last":1.0700000000000001}]},{"osd":6,"last update":"Wed Sep 18 10:33:18 2024","interfaces":[{"interface":"back","average":{"1min":1.0649999999999999,"5min":1.0649999999999999,"15min":1.0649999999999999},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":1.696,"5min":1.696,"15min":1.696},"last":1.1899999999999999},{"interface":"front","average":{"1min":1.1479999999999999,"5min":1.1479999999999999,"15min":1.1479999999999999},"min":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"max":{"1min":1.5289999999999999,"5min":1.5289999999999999,"15min":1.5289999999999999},"last":1.3169999999999999}]},{"osd":7,"last update":"Wed Sep 18 10:33:18 2024","interfaces":[{"interface":"back","average":{"1min":1.268,"5min":1.268,"15min":1.268},"min":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"max":{"1min":1.9350000000000001,"5min":1.9350000000000001,"15min":1.9350000000000001},"last":1.252},{"interface":"front","average":{"1min":1.244,"5min":1.244,"15min":1.244},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":2.6419999999999999,"5min":2.6419999999999999,"15min":2.6419999999999999},"last":1.6279999999999999}]},{"osd":8,"last update":"Wed Sep 18 10:33:18 2024","interfaces":[{"interface":"back","average":{"1min":1.306,"5min":1.306,"15min":1.306},"min":{"1min":1,"5min":1,"15min":1},"max":{"1min":1.613,"5min":1.613,"15min":1.613},"last":1.169},{"interface":"front","average":{"1min":1.278,"5min":1.278,"15min":1.278},"min":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"max":{"1min":1.9830000000000001,"5min":1.9830000000000001,"15min":1.9830000000000001},"last":1.4079999999999999}]},{"osd":9,"last update":"Wed Sep 18 10:33:18 2024","interfaces":[{"interface":"back","average":{"1min":1.2210000000000001,"5min":1.2210000000000001,"15min":1.2210000000000001},"min":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"max":{"1min":2.4220000000000002,"5min":2.4220000000000002,"15min":2.4220000000000002},"last":1.3779999999999999},{"interface":"front","average":{"1min":1.337,"5min":1.337,"15min":1.337},"min":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"max":{"1min":2.1349999999999998,"5min":2.1349999999999998,"15min":2.1349999999999998},"last":1.2829999999999999}]}]},{"osd":1,"up_from":59,"seq":253403070494,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":298028,"kb_used_data":676,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":104559572,"statfs":{"total":107374182400,"available":107069001728,"internally_reserved":0,"allocated":692224,"data_stored":332790,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,2,3,4,5,6,7,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Sep 18 10:33:17 2024","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":1.0980000000000001,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"min":{"1min":0.248,"5min":0.248,"15min":0.248},"max":{"1min":1.3,"5min":1.3,"15min":1.3},"last":0.85599999999999998}]},{"osd":2,"last update":"Wed Sep 18 10:33:17 2024","interfaces":[{"interface":"back","average":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.206,"5min":1.206,"15min":1.206},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":1.54,"5min":1.54,"15min":1.54},"last":0.81699999999999995}]},{"osd":3,"last update":"Wed Sep 18 10:33:17 2024","interfaces":[{"interface":"back","average":{"1min":1.129,"5min":1.129,"15min":1.129},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":5.7569999999999997,"5min":5.7569999999999997,"15min":5.7569999999999997},"last":1.248},{"interface":"front","average":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":2.9830000000000001,"5min":2.9830000000000001,"15min":2.9830000000000001},"last":1.718}]},{"osd":4,"last update":"Wed Sep 18 10:33:17 2024","interfaces":[{"interface":"back","average":{"1min":1.1000000000000001,"5min":1.1000000000000001,"15min":1.1000000000000001},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":3.8700000000000001,"5min":3.8700000000000001,"15min":3.8700000000000001},"last":1.3140000000000001},{"interface":"front","average":{"1min":1.1359999999999999,"5min":1.1359999999999999,"15min":1.1359999999999999},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":2.6520000000000001,"5min":2.6520000000000001,"15min":2.6520000000000001},"last":1.6870000000000001}]},{"osd":5,"last update":"Wed Sep 18 10:33:17 2024","interfaces":[{"interface":"back","average":{"1min":1.1160000000000001,"5min":1.1160000000000001,"15min":1.1160000000000001},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":2.0470000000000002,"5min":2.0470000000000002,"15min":2.0470000000000002},"last":1.792},{"interface":"front","average":{"1min":1.0920000000000001,"5min":1.0920000000000001,"15min":1.0920000000000001},"min":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"max":{"1min":1.6679999999999999,"5min":1.6679999999999999,"15min":1.6679999999999999},"last":0.97599999999999998}]},{"osd":6,"last update":"Wed Sep 18 10:33:17 2024","interfaces":[{"interface":"back","average":{"1min":1.3280000000000001,"5min":1.3280000000000001,"15min":1.3280000000000001},"min":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":3.1299999999999999,"5min":3.1299999999999999,"15min":3.1299999999999999},"last":1.0580000000000001},{"interface":"front","average":{"1min":1.123,"5min":1.123,"15min":1.123},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.9390000000000001,"5min":1.9390000000000001,"15min":1.9390000000000001},"last":1.4490000000000001}]},{"osd":7,"last update":"Wed Sep 18 10:33:17 2024","interfaces":[{"interface":"back","average":{"1min":1.1060000000000001,"5min":1.1060000000000001,"15min":1.1060000000000001},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":2.4900000000000002,"5min":2.4900000000000002,"15min":2.4900000000000002},"last":1.5429999999999999},{"interface":"front","average":{"1min":1.3120000000000001,"5min":1.3120000000000001,"15min":1.3120000000000001},"min":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"max":{"1min":3.2810000000000001,"5min":3.2810000000000001,"15min":3.2810000000000001},"last":1.573}]},{"osd":8,"last update":"Wed Sep 18 10:33:17 2024","interfaces":[{"interface":"back","average":{"1min":1.258,"5min":1.258,"15min":1.258},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":4.4969999999999999,"5min":4.4969999999999999,"15min":4.4969999999999999},"last":1.615},{"interface":"front","average":{"1min":1.1419999999999999,"5min":1.1419999999999999,"15min":1.1419999999999999},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":2.1629999999999998,"5min":2.1629999999999998,"15min":2.1629999999999998},"last":1.643}]},{"osd":9,"last update":"Wed Sep 18 10:33:17 2024","interfaces":[{"interface":"back","average":{"1min":1.4419999999999999,"5min":1.4419999999999999,"15min":1.4419999999999999},"min":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"max":{"1min":3.2160000000000002,"5min":3.2160000000000002,"15min":3.2160000000000002},"last":1.742},{"interface":"front","average":{"1min":1.2410000000000001,"5min":1.2410000000000001,"15min":1.2410000000000001},"min":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"max":{"1min":1.9430000000000001,"5min":1.9430000000000001,"15min":1.9430000000000001},"last":1.5109999999999999}]}]},{"osd":0,"up_from":59,"seq":253403070494,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":298036,"kb_used_data":672,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":104559564,"statfs":{"total":107374182400,"available":107068993536,"internally_reserved":0,"allocated":688128,"data_stored":332771,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[1,2,3,4,5,6,7,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Sep 18 10:33:15 2024","interfaces":[{"interface":"back","average":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":1.5880000000000001,"5min":1.5880000000000001,"15min":1.5880000000000001},"last":0.81799999999999995},{"interface":"front","average":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.917,"5min":1.917,"15min":1.917},"last":1.115}]},{"osd":2,"last update":"Wed Sep 18 10:33:15 2024","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.746,"15min":0.746},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.454,"5min":1.454,"15min":1.454},"last":0.97799999999999998},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"min":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.222,"5min":1.222,"15min":1.222},"last":0.77100000000000002}]},{"osd":3,"last update":"Wed Sep 18 10:33:15 2024","interfaces":[{"interface":"back","average":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":1.5509999999999999,"5min":1.5509999999999999,"15min":1.5509999999999999},"last":0.93000000000000005},{"interface":"front","average":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.6339999999999999,"5min":1.6339999999999999,"15min":1.6339999999999999},"last":1.208}]},{"osd":4,"last update":"Wed Sep 18 10:33:15 2024","interfaces":[{"interface":"back","average":{"1min":1,"5min":1,"15min":1},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":1.6339999999999999,"5min":1.6339999999999999,"15min":1.6339999999999999},"last":1.0660000000000001},{"interface":"front","average":{"1min":1.093,"5min":1.093,"15min":1.093},"min":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.704,"5min":1.704,"15min":1.704},"last":1.4570000000000001}]},{"osd":5,"last update":"Wed Sep 18 10:33:15 2024","interfaces":[{"interface":"back","average":{"1min":1.1699999999999999,"5min":1.1699999999999999,"15min":1.1699999999999999},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.9590000000000001,"5min":1.9590000000000001,"15min":1.9590000000000001},"last":1.532},{"interface":"front","average":{"1min":1.2050000000000001,"5min":1.2050000000000001,"15min":1.2050000000000001},"min":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"max":{"1min":2.206,"5min":2.206,"15min":2.206},"last":1.1779999999999999}]},{"osd":6,"last update":"Wed Sep 18 10:33:15 2024","interfaces":[{"interface":"back","average":{"1min":1.157,"5min":1.157,"15min":1.157},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":1.74,"5min":1.74,"15min":1.74},"last":1.6379999999999999},{"interface":"front","average":{"1min":1.135,"5min":1.135,"15min":1.135},"min":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"max":{"1min":2.8140000000000001,"5min":2.8140000000000001,"15min":2.8140000000000001},"last":1.248}]},{"osd":7,"last update":"Wed Sep 18 10:33:15 2024","interfaces":[{"interface":"back","average":{"1min":1.268,"5min":1.268,"15min":1.268},"min":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"max":{"1min":2.004,"5min":2.004,"15min":2.004},"last":1.5780000000000001},{"interface":"front","average":{"1min":1.357,"5min":1.357,"15min":1.357},"min":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"max":{"1min":2.7250000000000001,"5min":2.7250000000000001,"15min":2.7250000000000001},"last":1.706}]},{"osd":8,"last update":"Wed Sep 18 10:33:15 2024","interfaces":[{"interface":"back","average":{"1min":1.3049999999999999,"5min":1.3049999999999999,"15min":1.3049999999999999},"min":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"max":{"1min":2.1419999999999999,"5min":2.1419999999999999,"15min":2.1419999999999999},"last":1.661},{"interface":"front","average":{"1min":1.3779999999999999,"5min":1.3779999999999999,"15min":1.3779999999999999},"min":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"max":{"1min":2.8980000000000001,"5min":2.8980000000000001,"15min":2.8980000000000001},"last":1.7789999999999999}]},{"osd":9,"last update":"Wed Sep 18 10:33:15 2024","interfaces":[{"interface":"back","average":{"1min":1.2989999999999999,"5min":1.2989999999999999,"15min":1.2989999999999999},"min":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"max":{"1min":2.3109999999999999,"5min":2.3109999999999999,"15min":2.3109999999999999},"last":1.3120000000000001},{"interface":"front","average":{"1min":1.333,"5min":1.333,"15min":1.333},"min":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":2.2639999999999998,"5min":2.2639999999999998,"15min":2.2639999999999998},"last":1.7290000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-18T10:34:35.082 DEBUG:teuthology.orchestra.run.smithi134:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-09-18T10:34:35.402 INFO:teuthology.misc:Shutting down mds daemons... 2024-09-18T10:34:35.402 INFO:teuthology.misc:Shutting down osd daemons... 2024-09-18T10:34:35.403 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-09-18T10:34:35.403 INFO:teuthology.orchestra.run:waiting for 300 2024-09-18T10:34:35.488 INFO:tasks.ceph.osd.0:Stopped 2024-09-18T10:34:35.488 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-09-18T10:34:35.488 INFO:teuthology.orchestra.run:waiting for 300 2024-09-18T10:34:35.575 INFO:tasks.ceph.osd.1:Stopped 2024-09-18T10:34:35.575 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-09-18T10:34:35.575 INFO:teuthology.orchestra.run:waiting for 300 2024-09-18T10:34:35.643 INFO:tasks.ceph.osd.2:Stopped 2024-09-18T10:34:35.643 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-09-18T10:34:35.643 INFO:teuthology.orchestra.run:waiting for 300 2024-09-18T10:34:35.705 INFO:tasks.ceph.osd.3:Stopped 2024-09-18T10:34:35.706 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2024-09-18T10:34:35.706 INFO:teuthology.orchestra.run:waiting for 300 2024-09-18T10:34:35.783 INFO:tasks.ceph.osd.4:Stopped 2024-09-18T10:34:35.784 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2024-09-18T10:34:35.784 INFO:teuthology.orchestra.run:waiting for 300 2024-09-18T10:34:35.857 INFO:tasks.ceph.osd.5:Stopped 2024-09-18T10:34:35.857 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2024-09-18T10:34:35.857 INFO:teuthology.orchestra.run:waiting for 300 2024-09-18T10:34:35.935 INFO:tasks.ceph.osd.6:Stopped 2024-09-18T10:34:35.935 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2024-09-18T10:34:35.935 INFO:teuthology.orchestra.run:waiting for 300 2024-09-18T10:34:36.019 INFO:tasks.ceph.osd.7:Stopped 2024-09-18T10:34:36.019 DEBUG:tasks.ceph.osd.8:waiting for process to exit 2024-09-18T10:34:36.019 INFO:teuthology.orchestra.run:waiting for 300 2024-09-18T10:34:36.099 INFO:tasks.ceph.osd.8:Stopped 2024-09-18T10:34:36.100 DEBUG:tasks.ceph.osd.9:waiting for process to exit 2024-09-18T10:34:36.100 INFO:teuthology.orchestra.run:waiting for 300 2024-09-18T10:34:36.169 INFO:tasks.ceph.osd.9:Stopped 2024-09-18T10:34:36.169 INFO:teuthology.misc:Shutting down mgr daemons... 2024-09-18T10:34:36.169 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-09-18T10:34:36.169 INFO:teuthology.orchestra.run:waiting for 300 2024-09-18T10:34:36.202 INFO:tasks.ceph.mgr.x:Stopped 2024-09-18T10:34:36.202 INFO:teuthology.misc:Shutting down mon daemons... 2024-09-18T10:34:36.202 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-09-18T10:34:36.202 INFO:teuthology.orchestra.run:waiting for 300 2024-09-18T10:34:36.217 INFO:tasks.ceph.mon.a:Stopped 2024-09-18T10:34:36.218 INFO:tasks.ceph:Checking cluster log for badness... 2024-09-18T10:34:36.218 DEBUG:teuthology.orchestra.run.smithi134:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v 'but it is still running' | egrep -v 'overall HEALTH_' | egrep -v '\(OSDMAP_FLAGS\)' | egrep -v '\(OSD_' | egrep -v '\(PG_' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | head -n 1 2024-09-18T10:34:36.233 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi134.front.sepia.ceph.com 2024-09-18T10:34:36.234 DEBUG:teuthology.orchestra.run.smithi134:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-09-18T10:34:36.566 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi134.front.sepia.ceph.com 2024-09-18T10:34:36.566 DEBUG:teuthology.orchestra.run.smithi134:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-09-18T10:34:36.891 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi134.front.sepia.ceph.com 2024-09-18T10:34:36.892 DEBUG:teuthology.orchestra.run.smithi134:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-09-18T10:34:37.069 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi134.front.sepia.ceph.com 2024-09-18T10:34:37.069 DEBUG:teuthology.orchestra.run.smithi134:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-09-18T10:34:37.271 INFO:tasks.ceph:Archiving mon data... 2024-09-18T10:34:37.272 DEBUG:teuthology.misc:Transferring archived files from smithi134:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/skanta-2024-09-18_09:13:00-rados-wip-bharath11-testing-2024-09-18-1115-quincy-distro-default-smithi/7910878/data/mon.a.tgz 2024-09-18T10:34:37.272 DEBUG:teuthology.orchestra.run.smithi134:> mktemp 2024-09-18T10:34:37.277 INFO:teuthology.orchestra.run.smithi134.stdout:/tmp/tmp.6stRuIevix 2024-09-18T10:34:37.277 DEBUG:teuthology.orchestra.run.smithi134:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.6stRuIevix 2024-09-18T10:34:37.380 DEBUG:teuthology.orchestra.run.smithi134:> sudo chmod 0666 /tmp/tmp.6stRuIevix 2024-09-18T10:34:37.407 DEBUG:teuthology.orchestra.remote:smithi134:/tmp/tmp.6stRuIevix is 194KB 2024-09-18T10:34:37.424 DEBUG:teuthology.orchestra.run.smithi134:> rm -fr /tmp/tmp.6stRuIevix 2024-09-18T10:34:37.430 INFO:tasks.ceph:Cleaning ceph cluster... 2024-09-18T10:34:37.430 DEBUG:teuthology.orchestra.run.smithi134:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-09-18T10:34:37.487 INFO:tasks.ceph:Checking for errors in any valgrind logs... 2024-09-18T10:34:37.488 DEBUG:teuthology.orchestra.run.smithi134:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2024-09-18T10:34:37.515 INFO:teuthology.orchestra.run.smithi134.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2024-09-18T10:34:37.522 INFO:tasks.ceph:Archiving crash dumps... 2024-09-18T10:34:37.524 DEBUG:teuthology.misc:Transferring archived files from smithi134:/var/lib/ceph/crash to /home/teuthworker/archive/skanta-2024-09-18_09:13:00-rados-wip-bharath11-testing-2024-09-18-1115-quincy-distro-default-smithi/7910878/remote/smithi134/crash 2024-09-18T10:34:37.525 DEBUG:teuthology.orchestra.run.smithi134:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-09-18T10:34:37.539 INFO:tasks.ceph:Compressing logs... 2024-09-18T10:34:37.540 DEBUG:teuthology.orchestra.run.smithi134:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-18T10:34:37.610 INFO:teuthology.orchestra.run.smithi134.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33142.log 2024-09-18T10:34:37.622 INFO:teuthology.orchestra.run.smithi134.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31721.log 2024-09-18T10:34:37.624 INFO:teuthology.orchestra.run.smithi134.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32791.log 2024-09-18T10:34:37.625 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/ceph-client.admin.33142.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33142.log.gz 2024-09-18T10:34:37.625 INFO:teuthology.orchestra.run.smithi134.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32005.log 2024-09-18T10:34:37.625 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/ceph-client.admin.31721.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31721.log.gz 2024-09-18T10:34:37.625 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/ceph-client.admin.32791.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33250.log 2024-09-18T10:34:37.625 INFO:teuthology.orchestra.run.smithi134.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32791.log.gz 2024-09-18T10:34:37.625 INFO:teuthology.orchestra.run.smithi134.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30820.log 2024-09-18T10:34:37.625 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/ceph-client.admin.32005.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32005.log.gz 2024-09-18T10:34:37.625 INFO:teuthology.orchestra.run.smithi134.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2024-09-18T10:34:37.626 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/ceph-client.admin.33250.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33250.log.gz 2024-09-18T10:34:37.626 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/ceph-client.admin.30820.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33412.log 2024-09-18T10:34:37.626 INFO:teuthology.orchestra.run.smithi134.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30820.log.gz 2024-09-18T10:34:37.626 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31851.log 2024-09-18T10:34:37.626 INFO:teuthology.orchestra.run.smithi134.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32676.log 2024-09-18T10:34:37.626 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/ceph-client.admin.33412.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33412.log.gz 2024-09-18T10:34:37.626 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/ceph-client.admin.31851.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31542.log 2024-09-18T10:34:37.626 INFO:teuthology.orchestra.run.smithi134.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31851.log.gz 2024-09-18T10:34:37.626 INFO:teuthology.orchestra.run.smithi134.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33385.log 2024-09-18T10:34:37.626 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/ceph-client.admin.32676.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32676.log.gz 2024-09-18T10:34:37.626 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/ceph-client.admin.31542.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31542.log.gz 2024-09-18T10:34:37.626 INFO:teuthology.orchestra.run.smithi134.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30780.log 2024-09-18T10:34:37.627 INFO:teuthology.orchestra.run.smithi134.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25444.log 2024-09-18T10:34:37.627 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/ceph-client.admin.33385.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31904.log 2024-09-18T10:34:37.627 INFO:teuthology.orchestra.run.smithi134.stderr: 0.0%/var/log/ceph/ceph-client.admin.30780.log: -- replaced with /var/log/ceph/ceph-client.admin.33385.log.gz 2024-09-18T10:34:37.627 INFO:teuthology.orchestra.run.smithi134.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30780.log.gz 2024-09-18T10:34:37.627 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/ceph-client.admin.25444.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25472.log 2024-09-18T10:34:37.627 INFO:teuthology.orchestra.run.smithi134.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.25444.log.gz 2024-09-18T10:34:37.627 INFO:teuthology.orchestra.run.smithi134.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30382.log 2024-09-18T10:34:37.627 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/ceph-client.admin.31904.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31904.log.gz 2024-09-18T10:34:37.627 INFO:teuthology.orchestra.run.smithi134.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32238.log 2024-09-18T10:34:37.627 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/ceph-client.admin.25472.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.25472.log.gz 2024-09-18T10:34:37.627 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/ceph-client.admin.30382.log: 0.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.30382.log.gz 2024-09-18T10:34:37.627 INFO:teuthology.orchestra.run.smithi134.stderr:-5 --verbose -- /var/log/ceph/ceph-client.admin.30356.log 2024-09-18T10:34:37.627 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/ceph-client.admin.32238.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32899.log 2024-09-18T10:34:37.627 INFO:teuthology.orchestra.run.smithi134.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32238.log.gz 2024-09-18T10:34:37.628 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/ceph-client.admin.30356.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30818.log 2024-09-18T10:34:37.628 INFO:teuthology.orchestra.run.smithi134.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30356.log.gz 2024-09-18T10:34:37.647 INFO:teuthology.orchestra.run.smithi134.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33115.log 2024-09-18T10:34:37.648 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/ceph-client.admin.32899.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32899.log.gz 2024-09-18T10:34:37.648 INFO:teuthology.orchestra.run.smithi134.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32315.log 2024-09-18T10:34:37.648 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/ceph-client.admin.30818.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30818.log.gz 2024-09-18T10:34:37.648 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/ceph-client.admin.33115.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33115.log.gz 2024-09-18T10:34:37.648 INFO:teuthology.orchestra.run.smithi134.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31929.log 2024-09-18T10:34:37.649 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/ceph-client.admin.32315.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32315.log.gz 2024-09-18T10:34:37.649 INFO:teuthology.orchestra.run.smithi134.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25365.log 2024-09-18T10:34:37.649 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/ceph-client.admin.31929.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32572.log 2024-09-18T10:34:37.649 INFO:teuthology.orchestra.run.smithi134.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31929.log.gz 2024-09-18T10:34:37.649 INFO:teuthology.orchestra.run.smithi134.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30545.log 2024-09-18T10:34:37.649 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/ceph-client.admin.25365.log: /var/log/ceph/ceph-client.admin.32572.log: 0.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.25365.log.gz-5 2024-09-18T10:34:37.650 INFO:teuthology.orchestra.run.smithi134.stderr:--verbose -- /var/log/ceph/ceph-client.admin.25390.log 2024-09-18T10:34:37.650 INFO:teuthology.orchestra.run.smithi134.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32572.log.gz 2024-09-18T10:34:37.650 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/ceph-client.admin.30545.log: 0.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.30545.log.gz-5 2024-09-18T10:34:37.650 INFO:teuthology.orchestra.run.smithi134.stderr:--verbose -- /var/log/ceph/ceph-client.admin.30800.log 2024-09-18T10:34:37.650 INFO:teuthology.orchestra.run.smithi134.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31067.log 2024-09-18T10:34:37.650 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/ceph-client.admin.25390.log: /var/log/ceph/ceph-client.admin.30800.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30800.log.gz 2024-09-18T10:34:37.650 INFO:teuthology.orchestra.run.smithi134.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.25390.log.gz 2024-09-18T10:34:37.650 INFO:teuthology.orchestra.run.smithi134.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31064.log 2024-09-18T10:34:37.650 INFO:teuthology.orchestra.run.smithi134.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32263.log 2024-09-18T10:34:37.650 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/ceph-client.admin.31067.log: /var/log/ceph/ceph-client.admin.31064.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31067.log.gz 2024-09-18T10:34:37.650 INFO:teuthology.orchestra.run.smithi134.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31799.log 2024-09-18T10:34:37.651 INFO:teuthology.orchestra.run.smithi134.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31064.log.gz 2024-09-18T10:34:37.651 INFO:teuthology.orchestra.run.smithi134.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25201.log 2024-09-18T10:34:37.651 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/ceph-client.admin.32263.log: /var/log/ceph/ceph-client.admin.31799.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32263.log.gzgzip 2024-09-18T10:34:37.651 INFO:teuthology.orchestra.run.smithi134.stderr:-5 --verbose -- /var/log/ceph/ceph-client.admin.33493.log 2024-09-18T10:34:37.651 INFO:teuthology.orchestra.run.smithi134.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31799.log.gz 2024-09-18T10:34:37.651 INFO:teuthology.orchestra.run.smithi134.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33304.log 2024-09-18T10:34:37.651 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/ceph-client.admin.25201.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.25201.log.gz 2024-09-18T10:34:37.651 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/ceph-client.admin.33493.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31517.log 2024-09-18T10:34:37.651 INFO:teuthology.orchestra.run.smithi134.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33493.log.gz 2024-09-18T10:34:37.651 INFO:teuthology.orchestra.run.smithi134.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32367.log 2024-09-18T10:34:37.652 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/ceph-client.admin.33304.log: /var/log/ceph/ceph-client.admin.31517.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31517.log.gz 2024-09-18T10:34:37.652 INFO:teuthology.orchestra.run.smithi134.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.17236.log 2024-09-18T10:34:37.652 INFO:teuthology.orchestra.run.smithi134.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33304.log.gz 2024-09-18T10:34:37.652 INFO:teuthology.orchestra.run.smithi134.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31695.log 2024-09-18T10:34:37.652 INFO:teuthology.orchestra.run.smithi134.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2024-09-18T10:34:37.652 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/ceph-client.admin.32367.log: /var/log/ceph/ceph.tmp-client.admin.17236.log: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.17236.log.gz 2024-09-18T10:34:37.652 INFO:teuthology.orchestra.run.smithi134.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32367.log.gz 2024-09-18T10:34:37.652 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/ceph-client.admin.31695.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30544.log 2024-09-18T10:34:37.652 INFO:teuthology.orchestra.run.smithi134.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31695.log.gz 2024-09-18T10:34:37.652 INFO:teuthology.orchestra.run.smithi134.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30530.log 2024-09-18T10:34:37.652 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32057.log 2024-09-18T10:34:37.652 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/ceph-client.admin.30544.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30544.log.gz 2024-09-18T10:34:37.652 INFO:teuthology.orchestra.run.smithi134.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30530.log: /var/log/ceph/ceph-client.admin.31068.log 2024-09-18T10:34:37.652 INFO:teuthology.orchestra.run.smithi134.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30530.log.gz 2024-09-18T10:34:37.653 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/ceph-client.admin.32057.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32057.log.gz 2024-09-18T10:34:37.653 INFO:teuthology.orchestra.run.smithi134.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32135.log 2024-09-18T10:34:37.653 INFO:teuthology.orchestra.run.smithi134.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33007.log 2024-09-18T10:34:37.653 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/ceph-client.admin.31068.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31068.log.gz 2024-09-18T10:34:37.653 INFO:teuthology.orchestra.run.smithi134.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32598.log 2024-09-18T10:34:37.653 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/ceph-client.admin.32135.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32135.log.gz 2024-09-18T10:34:37.653 INFO:teuthology.orchestra.run.smithi134.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33088.log 2024-09-18T10:34:37.653 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/ceph-client.admin.33007.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33007.log.gz 2024-09-18T10:34:37.653 INFO:teuthology.orchestra.run.smithi134.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2024-09-18T10:34:37.653 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/ceph-client.admin.32598.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32598.log.gz 2024-09-18T10:34:37.653 INFO:teuthology.orchestra.run.smithi134.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32031.log 2024-09-18T10:34:37.654 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/ceph-client.admin.33088.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33088.log.gz 2024-09-18T10:34:37.654 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30482.log 2024-09-18T10:34:37.654 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/ceph-client.admin.32031.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25417.log 2024-09-18T10:34:37.654 INFO:teuthology.orchestra.run.smithi134.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32031.log.gz 2024-09-18T10:34:37.654 INFO:teuthology.orchestra.run.smithi134.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32289.log 2024-09-18T10:34:37.654 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/ceph-client.admin.30482.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30482.log.gz 2024-09-18T10:34:37.654 INFO:teuthology.orchestra.run.smithi134.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-09-18T10:34:37.654 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/ceph-client.admin.25417.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.25417.log.gz 2024-09-18T10:34:37.654 INFO:teuthology.orchestra.run.smithi134.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31346.log 2024-09-18T10:34:37.654 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/ceph-client.admin.32289.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32289.log.gz 2024-09-18T10:34:37.654 INFO:teuthology.orchestra.run.smithi134.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33196.log 2024-09-18T10:34:37.654 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30802.log 2024-09-18T10:34:37.655 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/ceph-client.admin.31346.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31346.log.gz 2024-09-18T10:34:37.655 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/ceph-client.admin.33196.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32521.log 2024-09-18T10:34:37.655 INFO:teuthology.orchestra.run.smithi134.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33196.log.gz 2024-09-18T10:34:37.655 INFO:teuthology.orchestra.run.smithi134.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31979.log 2024-09-18T10:34:37.655 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/ceph-client.admin.30802.log: 88.7% 0.0% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-09-18T10:34:37.655 INFO:teuthology.orchestra.run.smithi134.stderr: -- replaced with /var/log/ceph/ceph-client.admin.30802.log.gz 2024-09-18T10:34:37.655 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/ceph-client.admin.32521.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.6.log 2024-09-18T10:34:37.655 INFO:teuthology.orchestra.run.smithi134.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32521.log.gz 2024-09-18T10:34:37.655 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/ceph-client.admin.31979.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32953.log 2024-09-18T10:34:37.655 INFO:teuthology.orchestra.run.smithi134.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31979.log.gz 2024-09-18T10:34:37.655 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33331.log 2024-09-18T10:34:37.655 INFO:teuthology.orchestra.run.smithi134.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33466.log 2024-09-18T10:34:37.655 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/ceph-client.admin.32953.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32953.log.gz 2024-09-18T10:34:37.655 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/ceph-client.admin.33331.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25340.log 2024-09-18T10:34:37.655 INFO:teuthology.orchestra.run.smithi134.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33331.log.gz 2024-09-18T10:34:37.656 INFO:teuthology.orchestra.run.smithi134.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31075.log 2024-09-18T10:34:37.656 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/ceph-client.admin.33466.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33466.log.gz 2024-09-18T10:34:37.656 INFO:teuthology.orchestra.run.smithi134.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32737.log 2024-09-18T10:34:37.656 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/ceph-client.admin.25340.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.25340.log.gz 2024-09-18T10:34:37.656 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/ceph-client.admin.31075.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31075.log.gz 2024-09-18T10:34:37.656 INFO:teuthology.orchestra.run.smithi134.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30538.log 2024-09-18T10:34:37.656 INFO:teuthology.orchestra.run.smithi134.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31824.log 2024-09-18T10:34:37.656 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/ceph-client.admin.32737.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32737.log.gz 2024-09-18T10:34:37.656 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/ceph-client.admin.30538.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30538.log.gz 2024-09-18T10:34:37.662 INFO:teuthology.orchestra.run.smithi134.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.9.log 2024-09-18T10:34:37.662 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/ceph-client.admin.31824.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31069.log 2024-09-18T10:34:37.662 INFO:teuthology.orchestra.run.smithi134.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31824.log.gz 2024-09-18T10:34:37.662 INFO:teuthology.orchestra.run.smithi134.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32845.log 2024-09-18T10:34:37.662 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/ceph-osd.9.log: gzip /var/log/ceph/ceph-client.admin.31069.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.31747.log 2024-09-18T10:34:37.662 INFO:teuthology.orchestra.run.smithi134.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31069.log.gz 2024-09-18T10:34:37.663 INFO:teuthology.orchestra.run.smithi134.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31373.log 2024-09-18T10:34:37.663 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/ceph-client.admin.32845.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32845.log.gz 2024-09-18T10:34:37.663 INFO:teuthology.orchestra.run.smithi134.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30789.log 2024-09-18T10:34:37.663 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/ceph-client.admin.31747.log: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32701.log 2024-09-18T10:34:37.663 INFO:teuthology.orchestra.run.smithi134.stderr: -- replaced with /var/log/ceph/ceph-client.admin.31747.log.gz 2024-09-18T10:34:37.663 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/ceph-client.admin.31373.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31373.log.gz 2024-09-18T10:34:37.663 INFO:teuthology.orchestra.run.smithi134.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30817.log 2024-09-18T10:34:37.663 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/ceph-client.admin.30789.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30789.log.gz 2024-09-18T10:34:37.663 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/ceph-client.admin.32701.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32701.log.gz 2024-09-18T10:34:37.663 INFO:teuthology.orchestra.run.smithi134.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33572.log 2024-09-18T10:34:37.663 INFO:teuthology.orchestra.run.smithi134.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31082.log 2024-09-18T10:34:37.663 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/ceph-client.admin.30817.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30817.log.gz 2024-09-18T10:34:37.663 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/ceph-client.admin.33572.log: gzip 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33572.log.gz-5 2024-09-18T10:34:37.663 INFO:teuthology.orchestra.run.smithi134.stderr:--verbose -- /var/log/ceph/ceph-client.admin.31773.log 2024-09-18T10:34:37.663 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/ceph-client.admin.31082.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30778.log 2024-09-18T10:34:37.664 INFO:teuthology.orchestra.run.smithi134.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31082.log.gz 2024-09-18T10:34:37.664 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/ceph-client.admin.31773.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31773.log.gz 2024-09-18T10:34:37.664 INFO:teuthology.orchestra.run.smithi134.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2024-09-18T10:34:37.664 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/ceph-client.admin.30778.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30778.log.gz 2024-09-18T10:34:37.664 INFO:teuthology.orchestra.run.smithi134.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31319.log 2024-09-18T10:34:37.664 INFO:teuthology.orchestra.run.smithi134.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25948.log 2024-09-18T10:34:37.664 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/ceph-osd.1.log: /var/log/ceph/ceph-client.admin.31319.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31319.log.gz 2024-09-18T10:34:37.664 INFO:teuthology.orchestra.run.smithi134.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31670.log 2024-09-18T10:34:37.664 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/ceph-client.admin.25948.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.25948.log.gz 2024-09-18T10:34:37.664 INFO:teuthology.orchestra.run.smithi134.stderr: 91.5% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2024-09-18T10:34:37.664 INFO:teuthology.orchestra.run.smithi134.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30541.log 2024-09-18T10:34:37.664 INFO:teuthology.orchestra.run.smithi134.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32624.log 2024-09-18T10:34:37.664 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/ceph-client.admin.31670.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31670.log.gz 2024-09-18T10:34:37.665 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/ceph-client.admin.30541.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30541.log.gz 2024-09-18T10:34:37.667 INFO:teuthology.orchestra.run.smithi134.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30428.log 2024-09-18T10:34:37.667 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/ceph-client.admin.32624.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33547.log 2024-09-18T10:34:37.667 INFO:teuthology.orchestra.run.smithi134.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32624.log.gz 2024-09-18T10:34:37.667 INFO:teuthology.orchestra.run.smithi134.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31049.log 2024-09-18T10:34:37.667 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/ceph-client.admin.30428.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32444.log 2024-09-18T10:34:37.667 INFO:teuthology.orchestra.run.smithi134.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30428.log.gz 2024-09-18T10:34:37.667 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/ceph-client.admin.33547.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33547.log.gz 2024-09-18T10:34:37.668 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/ceph-client.admin.31049.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31049.log.gz 2024-09-18T10:34:37.672 INFO:teuthology.orchestra.run.smithi134.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31954.log 2024-09-18T10:34:37.672 INFO:teuthology.orchestra.run.smithi134.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31110.log 2024-09-18T10:34:37.672 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/ceph-client.admin.32444.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32444.log.gz 2024-09-18T10:34:37.673 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/ceph-client.admin.31954.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25266.log 2024-09-18T10:34:37.673 INFO:teuthology.orchestra.run.smithi134.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31954.log.gz 2024-09-18T10:34:37.673 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/ceph-client.admin.31110.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31488.log 2024-09-18T10:34:37.673 INFO:teuthology.orchestra.run.smithi134.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31110.log.gz 2024-09-18T10:34:37.673 INFO:teuthology.orchestra.run.smithi134.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30455.log 2024-09-18T10:34:37.673 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/ceph-client.admin.25266.log: /var/log/ceph/ceph-client.admin.31488.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32926.log 2024-09-18T10:34:37.673 INFO:teuthology.orchestra.run.smithi134.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.25266.log.gz 2024-09-18T10:34:37.673 INFO:teuthology.orchestra.run.smithi134.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31488.log.gz 2024-09-18T10:34:37.673 INFO:teuthology.orchestra.run.smithi134.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32341.log 2024-09-18T10:34:37.673 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/ceph-client.admin.30455.log: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32160.log 2024-09-18T10:34:37.673 INFO:teuthology.orchestra.run.smithi134.stderr: -- replaced with /var/log/ceph/ceph-client.admin.30455.log.gz 2024-09-18T10:34:37.673 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/ceph-client.admin.32926.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32926.log.gz 2024-09-18T10:34:37.673 INFO:teuthology.orchestra.run.smithi134.stderr:gzip /var/log/ceph/ceph-client.admin.32341.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.25653.log 2024-09-18T10:34:37.673 INFO:teuthology.orchestra.run.smithi134.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32341.log.gz 2024-09-18T10:34:37.673 INFO:teuthology.orchestra.run.smithi134.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32980.log 2024-09-18T10:34:37.674 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/ceph-client.admin.32160.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32160.log.gz 2024-09-18T10:34:37.674 INFO:teuthology.orchestra.run.smithi134.stderr:gzip /var/log/ceph/ceph-client.admin.25653.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.25526.log 2024-09-18T10:34:37.674 INFO:teuthology.orchestra.run.smithi134.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.25653.log.gz 2024-09-18T10:34:37.674 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/ceph-client.admin.32980.log: gzip -5 --verbose -- 0.0%/var/log/ceph/ceph-client.admin.31620.log 2024-09-18T10:34:37.674 INFO:teuthology.orchestra.run.smithi134.stderr: -- replaced with /var/log/ceph/ceph-client.admin.32980.log.gz 2024-09-18T10:34:37.674 INFO:teuthology.orchestra.run.smithi134.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25526.log: /var/log/ceph/ceph-client.admin.32872.log 2024-09-18T10:34:37.674 INFO:teuthology.orchestra.run.smithi134.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.25526.log.gz 2024-09-18T10:34:37.674 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/ceph-client.admin.31620.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31620.log.gz 2024-09-18T10:34:37.677 INFO:teuthology.orchestra.run.smithi134.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33358.log 2024-09-18T10:34:37.677 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/ceph-client.admin.32872.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32872.log.gz 2024-09-18T10:34:37.677 INFO:teuthology.orchestra.run.smithi134.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33439.log 2024-09-18T10:34:37.677 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/ceph-client.admin.33358.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33061.log 2024-09-18T10:34:37.677 INFO:teuthology.orchestra.run.smithi134.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33358.log.gz 2024-09-18T10:34:37.677 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/ceph-client.admin.33439.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31063.log 2024-09-18T10:34:37.678 INFO:teuthology.orchestra.run.smithi134.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33439.log.gz 2024-09-18T10:34:37.678 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/ceph-client.admin.33061.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25551.log 2024-09-18T10:34:37.678 INFO:teuthology.orchestra.run.smithi134.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33061.log.gz 2024-09-18T10:34:37.678 INFO:teuthology.orchestra.run.smithi134.stderr:gzip -5 --verbose /var/log/ceph/ceph-client.admin.31063.log: -- /var/log/ceph/ceph-client.admin.30795.log 2024-09-18T10:34:37.678 INFO:teuthology.orchestra.run.smithi134.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31063.log.gz 2024-09-18T10:34:37.678 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/ceph-client.admin.25551.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-09-18T10:34:37.678 INFO:teuthology.orchestra.run.smithi134.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.25551.log.gz 2024-09-18T10:34:37.678 INFO:teuthology.orchestra.run.smithi134.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32083.log 2024-09-18T10:34:37.678 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/ceph-client.admin.30795.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30795.log.gz 2024-09-18T10:34:37.678 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33169.log 2024-09-18T10:34:37.678 INFO:teuthology.orchestra.run.smithi134.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30542.log 2024-09-18T10:34:37.678 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/ceph-client.admin.32083.log: 0.0%/var/log/ceph/ceph-client.admin.33169.log: -- replaced with /var/log/ceph/ceph-client.admin.32083.log.gz 2024-09-18T10:34:37.678 INFO:teuthology.orchestra.run.smithi134.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33169.log.gz 2024-09-18T10:34:37.678 INFO:teuthology.orchestra.run.smithi134.stderr: 84.4% -- replaced with /var/log/ceph/ceph.log.gz 2024-09-18T10:34:37.680 INFO:teuthology.orchestra.run.smithi134.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25499.log 2024-09-18T10:34:37.680 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/ceph-client.admin.30542.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2024-09-18T10:34:37.680 INFO:teuthology.orchestra.run.smithi134.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30542.log.gz 2024-09-18T10:34:37.681 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/ceph-client.admin.25499.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32212.log 2024-09-18T10:34:37.681 INFO:teuthology.orchestra.run.smithi134.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.25499.log.gz 2024-09-18T10:34:37.682 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32818.log 2024-09-18T10:34:37.682 INFO:teuthology.orchestra.run.smithi134.stderr:gzip /var/log/ceph/ceph-client.admin.32212.log: -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2024-09-18T10:34:37.682 INFO:teuthology.orchestra.run.smithi134.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32212.log.gz 2024-09-18T10:34:37.682 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/ceph-client.admin.32818.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32818.log.gz 2024-09-18T10:34:37.682 INFO:teuthology.orchestra.run.smithi134.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30535.log 2024-09-18T10:34:37.692 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32418.log 2024-09-18T10:34:37.693 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/ceph-client.admin.30535.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30535.log.gz 2024-09-18T10:34:37.694 INFO:teuthology.orchestra.run.smithi134.stderr: 92.0% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2024-09-18T10:34:37.694 INFO:teuthology.orchestra.run.smithi134.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32393.log 2024-09-18T10:34:37.695 INFO:teuthology.orchestra.run.smithi134.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30536.log 2024-09-18T10:34:37.695 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/ceph-client.admin.32418.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32418.log.gz 2024-09-18T10:34:37.695 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/ceph-client.admin.32393.log: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33520.log 2024-09-18T10:34:37.696 INFO:teuthology.orchestra.run.smithi134.stderr: -- replaced with /var/log/ceph/ceph-client.admin.32393.log.gz 2024-09-18T10:34:37.698 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/ceph-client.admin.30536.log: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33277.log 2024-09-18T10:34:37.698 INFO:teuthology.orchestra.run.smithi134.stderr: -- replaced with /var/log/ceph/ceph-client.admin.30536.log.gz 2024-09-18T10:34:37.698 INFO:teuthology.orchestra.run.smithi134.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2024-09-18T10:34:37.698 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/ceph-client.admin.33520.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33520.log.gz 2024-09-18T10:34:37.698 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/ceph-client.admin.33277.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31645.log 2024-09-18T10:34:37.698 INFO:teuthology.orchestra.run.smithi134.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33277.log.gz 2024-09-18T10:34:37.698 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25579.log 2024-09-18T10:34:37.698 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/ceph-client.admin.31645.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31645.log.gz 2024-09-18T10:34:37.700 INFO:teuthology.orchestra.run.smithi134.stderr: 92.1%gzip -5 --verbose -- /var/log/ceph/ceph-osd.8.log 2024-09-18T10:34:37.700 INFO:teuthology.orchestra.run.smithi134.stderr: -- replaced with /var/log/ceph/ceph-osd.6.log.gz 2024-09-18T10:34:37.700 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/ceph-client.admin.25579.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.25579.log.gz 2024-09-18T10:34:37.701 INFO:teuthology.orchestra.run.smithi134.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32109.log 2024-09-18T10:34:37.704 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/ceph-osd.8.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33034.log 2024-09-18T10:34:37.705 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/ceph-client.admin.32109.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32109.log.gz 2024-09-18T10:34:37.712 INFO:teuthology.orchestra.run.smithi134.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32186.log 2024-09-18T10:34:37.712 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/ceph-client.admin.33034.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33034.log.gz 2024-09-18T10:34:37.716 INFO:teuthology.orchestra.run.smithi134.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30531.log 2024-09-18T10:34:37.717 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/ceph-client.admin.32186.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32650.log 2024-09-18T10:34:37.717 INFO:teuthology.orchestra.run.smithi134.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32186.log.gz 2024-09-18T10:34:37.717 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/ceph-client.admin.30531.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30531.log.gz 2024-09-18T10:34:37.720 INFO:teuthology.orchestra.run.smithi134.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32469.log 2024-09-18T10:34:37.720 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/ceph-client.admin.32650.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32650.log.gz 2024-09-18T10:34:37.720 INFO:teuthology.orchestra.run.smithi134.stderr: 92.3% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2024-09-18T10:34:37.724 INFO:teuthology.orchestra.run.smithi134.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32764.log 2024-09-18T10:34:37.724 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/ceph-client.admin.32469.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31595.log 2024-09-18T10:34:37.724 INFO:teuthology.orchestra.run.smithi134.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32469.log.gz 2024-09-18T10:34:37.724 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/ceph-client.admin.32764.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30543.log 2024-09-18T10:34:37.724 INFO:teuthology.orchestra.run.smithi134.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32764.log.gz 2024-09-18T10:34:37.724 INFO:teuthology.orchestra.run.smithi134.stderr: 92.1% -- replaced with /var/log/ceph/ceph-osd.9.log.gz 2024-09-18T10:34:37.724 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/ceph-client.admin.31595.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31081.log 2024-09-18T10:34:37.724 INFO:teuthology.orchestra.run.smithi134.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31595.log.gz 2024-09-18T10:34:37.724 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/ceph-client.admin.30543.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32494.log 2024-09-18T10:34:37.725 INFO:teuthology.orchestra.run.smithi134.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30543.log.gz 2024-09-18T10:34:37.725 INFO:teuthology.orchestra.run.smithi134.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30408.log 2024-09-18T10:34:37.725 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/ceph-client.admin.31081.log: /var/log/ceph/ceph-client.admin.32494.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.7.log 2024-09-18T10:34:37.725 INFO:teuthology.orchestra.run.smithi134.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31081.log.gz 2024-09-18T10:34:37.725 INFO:teuthology.orchestra.run.smithi134.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32494.log.gz 2024-09-18T10:34:37.725 INFO:teuthology.orchestra.run.smithi134.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31878.log 2024-09-18T10:34:37.725 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/ceph-client.admin.30408.log: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33223.log 2024-09-18T10:34:37.725 INFO:teuthology.orchestra.run.smithi134.stderr: -- replaced with /var/log/ceph/ceph-client.admin.30408.log.gz 2024-09-18T10:34:37.725 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/ceph-osd.7.log: /var/log/ceph/ceph-client.admin.31878.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31878.log.gz 2024-09-18T10:34:37.728 INFO:teuthology.orchestra.run.smithi134.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32546.log 2024-09-18T10:34:37.728 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/ceph-client.admin.33223.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2024-09-18T10:34:37.728 INFO:teuthology.orchestra.run.smithi134.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33223.log.gz 2024-09-18T10:34:37.729 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/ceph-client.admin.32546.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32546.log.gz 2024-09-18T10:34:37.736 INFO:teuthology.orchestra.run.smithi134.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30803.log 2024-09-18T10:34:37.736 INFO:teuthology.orchestra.run.smithi134.stderr:/var/log/ceph/ceph-osd.2.log: /var/log/ceph/ceph-client.admin.30803.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30803.log.gz 2024-09-18T10:34:37.746 INFO:teuthology.orchestra.run.smithi134.stderr: 91.9% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2024-09-18T10:34:37.759 INFO:teuthology.orchestra.run.smithi134.stderr: 92.2% -- replaced with /var/log/ceph/ceph-osd.8.log.gz 2024-09-18T10:34:37.759 INFO:teuthology.orchestra.run.smithi134.stderr: 92.2% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2024-09-18T10:34:37.764 INFO:teuthology.orchestra.run.smithi134.stderr: 92.2% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2024-09-18T10:34:37.764 INFO:teuthology.orchestra.run.smithi134.stderr: 91.5% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2024-09-18T10:34:37.772 INFO:teuthology.orchestra.run.smithi134.stderr: 92.1% -- replaced with /var/log/ceph/ceph-osd.7.log.gz 2024-09-18T10:34:37.775 INFO:teuthology.orchestra.run.smithi134.stderr: 92.3% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2024-09-18T10:34:37.777 INFO:teuthology.orchestra.run.smithi134.stderr: 2024-09-18T10:34:37.777 INFO:teuthology.orchestra.run.smithi134.stderr:real 0m0.189s 2024-09-18T10:34:37.777 INFO:teuthology.orchestra.run.smithi134.stderr:user 0m0.887s 2024-09-18T10:34:37.777 INFO:teuthology.orchestra.run.smithi134.stderr:sys 0m0.084s 2024-09-18T10:34:37.777 INFO:tasks.ceph:Archiving logs... 2024-09-18T10:34:37.777 DEBUG:teuthology.misc:Transferring archived files from smithi134:/var/log/ceph to /home/teuthworker/archive/skanta-2024-09-18_09:13:00-rados-wip-bharath11-testing-2024-09-18-1115-quincy-distro-default-smithi/7910878/remote/smithi134/log 2024-09-18T10:34:37.778 DEBUG:teuthology.orchestra.run.smithi134:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-18T10:34:38.009 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-09-18T10:34:38.020 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-09-18T10:34:38.020 DEBUG:teuthology.orchestra.run.smithi134:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-18T10:34:38.060 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-09-18T10:34:38.165 INFO:teuthology.task.clock:Checking final clock skew... 2024-09-18T10:34:38.165 DEBUG:teuthology.orchestra.run.smithi134:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-18T10:34:38.188 INFO:teuthology.orchestra.run.smithi134.stdout: remote refid st t when poll reach delay offset jitter 2024-09-18T10:34:38.188 INFO:teuthology.orchestra.run.smithi134.stdout:============================================================================== 2024-09-18T10:34:38.189 INFO:teuthology.orchestra.run.smithi134.stdout:+hv01.front.sepi 67.205.162.81 3 u 26 64 37 0.069 -0.299 0.604 2024-09-18T10:34:38.189 INFO:teuthology.orchestra.run.smithi134.stdout:*hv02.front.sepi 63.231.80.2 3 u 32 64 37 0.114 -1.269 1.102 2024-09-18T10:34:38.189 INFO:teuthology.orchestra.run.smithi134.stdout:+hv03.front.sepi 74.6.168.72 3 u 31 64 37 0.090 1.111 1.094 2024-09-18T10:34:38.189 INFO:teuthology.orchestra.run.smithi134.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2024-09-18T10:34:38.190 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-09-18T10:34:38.291 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-09-18T10:34:38.291 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-09-18T10:34:38.330 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-09-18T10:34:38.458 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-09-18T10:34:38.629 INFO:teuthology.task.internal:Duration was 637.225466 seconds 2024-09-18T10:34:38.629 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-09-18T10:34:38.783 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-09-18T10:34:38.783 DEBUG:teuthology.orchestra.run.smithi134:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-18T10:34:38.834 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-09-18T10:34:38.834 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi134.front.sepia.ceph.com 2024-09-18T10:34:38.834 DEBUG:teuthology.orchestra.run.smithi134:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-18T10:34:38.847 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-09-18T10:34:38.847 DEBUG:teuthology.orchestra.run.smithi134:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-18T10:34:38.875 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-09-18T10:34:38.875 DEBUG:teuthology.orchestra.run.smithi134:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-18T10:34:38.989 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-09-18T10:34:39.004 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-09-18T10:34:39.004 DEBUG:teuthology.orchestra.run.smithi134:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-18T10:34:39.017 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-09-18T10:34:39.042 DEBUG:teuthology.orchestra.run.smithi134:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-18T10:34:39.072 INFO:teuthology.orchestra.run.smithi134.stdout:kernel.core_pattern = core 2024-09-18T10:34:39.090 DEBUG:teuthology.orchestra.run.smithi134:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-18T10:34:39.094 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-18T10:34:39.094 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-09-18T10:34:39.104 INFO:teuthology.task.internal:Transferring archived files... 2024-09-18T10:34:39.105 DEBUG:teuthology.misc:Transferring archived files from smithi134:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/skanta-2024-09-18_09:13:00-rados-wip-bharath11-testing-2024-09-18-1115-quincy-distro-default-smithi/7910878/remote/smithi134 2024-09-18T10:34:39.105 DEBUG:teuthology.orchestra.run.smithi134:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-18T10:34:39.128 INFO:teuthology.task.internal:Removing archive directory... 2024-09-18T10:34:39.128 DEBUG:teuthology.orchestra.run.smithi134:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-18T10:34:39.173 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-09-18T10:34:39.183 INFO:teuthology.task.internal:Not uploading archives. 2024-09-18T10:34:39.183 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-09-18T10:34:39.303 INFO:teuthology.task.internal:Tidying up after the test... 2024-09-18T10:34:39.303 DEBUG:teuthology.orchestra.run.smithi134:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-18T10:34:39.310 INFO:teuthology.orchestra.run.smithi134.stdout: 924112 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 18 10:34 /home/ubuntu/cephtest 2024-09-18T10:34:39.312 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-09-18T10:34:39.416 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-09-18T10:34:39.625 INFO:teuthology.run:Summary data: description: rados/singleton-nomsgr/{all/health-warnings mon_election/classic rados supported-random-distro$/{ubuntu_latest}} duration: 637.2254664897919 flavor: default owner: scheduled_skanta@teuthology success: true 2024-09-18T10:34:39.626 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-18T10:34:39.719 INFO:teuthology.run:pass